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CHAPTER  5 


Simulation  Training  in  Health  Care 

Frank  A.  Drews  &  Jonathan  Z.  Bakdash 

In  this  chapter,  we  discuss  the  application  of  human  factors  and  ergonomics  to  developing 
effective  simulation  training  in  health  care.  Simulation  provides  a  safe,  effective  method 
for  training  and  assessing  human  performance.  In  aviation,  simulation-based  training  and 
assessment  has  been  widely  used,  significantly  improving  safety.  This  progress  would  have 
been  impossible  without  the  involvement  of  human  factors  and  ergonomics.  Although 
aviation  and  health  care  have  similarities,  there  also  are  differences  that  complicate  the 
widespread  implementation  of  simulation  in  health  care. 


Simulation  is  an  important  element  for  improving  patient  safety.  There  is  growing 
evidence  that  simulation  training  increases  adherence  to  best  practices,  improves 
clinical  outcomes,  and  reduces  the  costs  associated  with  care.  For  effective  simula¬ 
tion  training  in  health  care,  or  any  domain,  the  simulation  must  support  psychologi¬ 
cal  fidelity,  replicating  the  major  cognitive  operations  of  the  actual  or  real-world 
(clinical)  tasks.  To  achieve  psychological  fidelity,  simulation  development  must  be 
based  on  a  thorough  analysis  of  the  tasks  and  goals  present  in  the  clinical  environ¬ 
ment.  The  simulation  tool  is  selected  with  physical  realism  being  only  one  means  to 
support  the  psychological  fidelity  of  the  actual  clinical  task.  This  approach  maximizes 
the  efficacy  of  simulation  training  and  minimizes  costs. 

Additional  critical  challenges  to  widespread  implementation  of  effective  simula¬ 
tion  training  in  health  care  include  the  belief  that  high  physical  realism  is  necessary, 
a  lack  of  standardization  in  clinical  procedures  and  equipment,  the  absence  of  a 
mechanism  for  sharing  simulation  scenarios,  and  the  difficulties  associated  with 
measuring  clinical  outcomes  for  patients  with  the  consequent  issues  of  validity  and 
reliability  of  variables  used  as  inputs  in  a  cost-benefit  analysis  (CBA)  of  simulation 
training. 

With  the  advancement  of  technology  and  the  increasing  need  to  improve  human 
performance  in  complex  systems,  new  approaches  have  emerged  to  support  opera¬ 
tors  while  performing  complex  tasks.  One  of  these  approaches  is  simulation-based 
training.  Human  factors  and  ergonomics  is  an  important  contributor  to  the  creation  of 
simulation  based  training  programs  (Jones,  Hennessey,  &  Deutsch,  1985). 

Today,  simulations  are  used  for  diverse  purposes,  including  entertainment,  educa¬ 
tion,  training,  system  evaluation,  and  research.  They  are  commonly  used  in  domains 
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where  humans  are  required  to  manipulate  or  control  complex  systems,  including 
aviation,  health  care,  power  plants,  refineries,  and  many  others.  Some  simulators  are 
created  to  be  as  lifelike  as  possible  and  provide  an  experience  nearly  identical  to  the 
actual  system,  whereas  others  reproduce  only  the  key  characteristics  of  the  simu¬ 
lated  system.  Regardless  of  their  design  or  complexity,  simulators  are  used  in  situa¬ 
tions  when  use  of  the  actual  system  is  impractical  because  of  financial  limitations, 
system  availability,  ethical  reasons,  or  risk  of  life  (Haluck  &  Krummel,  2000;  Schlectre, 
Bessemer,  &  Kolosh,  1992;  Taylor,  Lintern,  &  Koonce,  1993). 

With  effective  training  fundamental  to  the  acquisition  of  the  skills  and  knowledge 
necessary  to  deliver  safe,  high-quality  health  care,  simulation  training  can  support 
these  goals.  Traditional  training  in  many  health  care  areas  involves  a  mentor-appren¬ 
tice  approach,  whereby  the  trainee  follows  an  experienced  trainer  and  receives  train¬ 
ing  with  actual  patients  (Haluck  &  Krummel,  2000;  Haque  &  Srinivasan,  2006).  This 
practice  is  reflective  of  the  still  common  training  approach  of  “see  one,  do  one,  teach 
one”  (Vozenilek,  Huff,  Reznek,  &  Gordon,  2004;  Ziv,  Wolpe,  Small,  &  Click,  2006). 
Gawande  (2007)  describes  the  consequences  and  the  challenges  of  this  approach. 

The  goal  of  this  chapter  is  to  provide  a  review  that  will  be  helpful  to  researchers 
currently  using  or  planning  to  use  simulation.  In  addition,  practitioners  interested  in 
using  a  theory-driven  and  empirically  supported  approach  to  improve  the  quality  of 
their  simulations  and  measures  to  assess  performance  will  benefit.  Finally,  human 
factors  engineers  and  ergonomists  with  an  interest  in  health  care  can  benefit  from 
reading  this  chapter  as  it  focuses  on  improvement  of  health  care  delivery,  one  of  the 
greatest  and  most  interesting  challenges  to  our  field. 

In  this  chapter,  we  review  the  current  literature  on  simulation  training  in  health 
care.  First,  we  provide  a  brief  overview  of  the  history  of  simulation,  ranging  from  avia¬ 
tion  to  health  care.  Second,  we  discuss  the  characteristics  of  simulation  and  the  types 
of  simulation  that  are  being  used  for  training  and  performance  evaluation.  This  topic 
leads  to  the  discussion  of  the  rationale  for  simulator  training  in  health  care.  In  the 
fourth  section  of  this  chapter,  we  will  analyze  what  makes  a  simulation  effective,  ana¬ 
lyzing  the  psychological  operations  of  the  task  with  regard  to  the  goals  that  are 
intended  to  be  accomplished.  In  the  fifth  section,  we  describe  the  issue  of  cost  analy¬ 
sis  in  the  context  of  simulation.  Last,  the  chapter  concludes  with  a  summary  of  the 
challenges  of  health  care  simulation  and  provides  an  outlook  into  the  future  of  simu¬ 
lation  in  health  care. 


A  BRIEF  HISTORY  OF  SIMULATION:  FROM  AVIATION  TO 

HEALTH  CARE 

The  Beginnings:  Simulation  in  Aviation 

With  the  advancement  of  aviation  at  the  beginning  of  the  last  century,  frequent 
accidents  of  novice  pilots  marked  the  need  for  a  better  and  safer  approach  toward 
training.  In  his  discussion  of  virtual  reality  surgical  simulation,  Satava  (2006,  p.  2) 
summarizes  this  motivation  well:  “Whenever  something  is  too  dangerous,  expensive, 
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or  distant  in  time,  place,  or  imagination  to  physically  experience,  there  have  been 
attempts  to  simulate  the  experience.” 

Moroney  and  Moroney  (1999)  and  Page  (2000)  provide  good  overviews  of  the  his¬ 
tory  of  aviation  simulation.  Using  these  two  references,  we  provide  a  brief  overview 
here.  At  the  beginning  of  the  20th  century,  interest  in  aviation  was  great,  but  training 
to  become  a  pilot  was  expensive  and  dangerous.  For  example,  early  training  required 
students  to  go  through  a  sequence  of  exercises.  After  initial  passenger  flights,  the 
student  would  attempt  taxiing  a  low-powered  machine  to  learn  control  of  the  rudder 
on  the  ground.  Next,  the  student  would  advance  to  a  plane  that  would  allow  small 
hops  involving  elevator  control,  which  then  would  lead  to  actual  flight. 

In  response  to  the  high  cost  of  flight  training,  alternate  means  for  instruction 
emerged.  Several  engineers  began  the  development  of  ground-based  aviation  train¬ 
ers,  which  were  essentially  aircraft  mounted  to  a  universal  joint  that  could  be  placed 
in  a  strong  prevailing  wind. 

As  a  result  of  the  high  demand  for  pilots  during  World  War  I,  the  discipline  of  avia¬ 
tion  psychology  emerged  with  the  goal  of  developing  tools  for  more  effective  selec¬ 
tion  of  future  pilots.  However,  it  was  not  until  Edwin  Link  developed  the  Link  Pilot 
Maker  around  1928  that  ground-based  aviation  simulators  became  widely  used. 

Link’s  Pilot  Maker  used  air  pressure  to  move  actuators  in  response  to  control 
movements  from  the  pilot,  therefore  removing  the  dependence  on  strong  winds  and 
manual  movement  of  the  previous  training  devices.  When  in  1934  the  U.S.  Army  was 
searching  for  ways  to  teach  instrument-only  flight  to  pilots,  it  identified  the  Pilot 
Maker  as  a  system  that  fulfilled  its  needs.  Within  a  few  years,  Link  Trainers  were  being 
sold  to  air  forces  across  Europe  and  in  Japan  and  became  an  invaluable  tool  to  teach 
pilots  to  fly  an  airplane  on  instruments  only  (Smith  &  Smith,  1989). 

The  advancement  of  simulators  took  a  leap  due  to  the  application  of  electrical  and 
electronic  methods.  Because  of  the  success  of  the  training  of  aviation  pilots  with  the 
Pilot  Maker,  other  simulators  were  developed  to  train  additional  crew  members. 
During  the  years  that  followed,  simulators  advanced  in  complexity,  increased  in  size, 
and  rose  in  cost.  Following  World  War  II,  computers  began  to  replace  the  levers, 
motors,  and  air  bellows  that  previously  controlled  simulators.  The  simulator  cockpits 
became  near  perfect  replications  of  actual  aircraft  cockpits  that  included  moving 
views  of  the  ground.  With  advances  in  the  technology  behind  the  simulators,  the 
scope  of  the  training  application  increased  as  well.  Simulators  are  now  used  to  train 
myriad  aspects  of  flight,  including:  basic  system  use  (Bell  &  Waag,  1998;  Stark,  1989), 
instrument  use  (Ortiz,  1994),  decision  making  (Connolly,  Blackwell,  &  Lester,  1989), 
and  emergency  procedures  (Bell  &  Waag,  1998).  The  effective  use  of  simulators  in  the 
field  of  aviation  provided  a  foundation  upon  which  other  domains  created  their  own 
simulation  programs  (Seymour  et  al.,  2002). 

Simulation  in  Health  Care 

Health  care  simulation  can  be  dated  back  to  the  development  of  the  part-task  simu¬ 
lator  Resusci-Anne  (for  training  parts  of  a  procedure)  by  the  Laerdal  Company  in  the 
1950s.  In  response  to  a  paper  on  the  effectiveness  of  mouth-to-mouth  ventilation 
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(Safar  &  McMahon,  1958),  Resusci-Anne  was  developed  to  support  skill  acquisition 
for  this  procedure.  With  Resusci-Anne,  Laerdal  created  a  full-size  training  mannequin 
that  allowed  training  on  clinical  scenarios,  like  obstructed  airways.  A  later  version  of 
this  simulator  included  a  spring  mechanism  in  the  chest  allowing  simulation  of  chest 
compressions  as  well. 

In  the  early  1960s,  standardized  or  simulated  patients  (trained  actors)  were  intro¬ 
duced  to  help  train  communication  (e.g.,  taking  a  patient  history)  and  clinical  skills 
(e.g.,  physical  examination)  (Rosen,  2008),  having  the  advantage  of  providing  consistent 
conditions  for  assessing  communication  and  clinical  skills  in  specific  situations  (Wallace, 
1997). 

The  next  step  in  health  care  simulation  was  the  development  of  a  computer-con- 
trolled  modern  simulator  (Abrahamson,  Denson,  &  Wolf,  1969)  in  the  late  1960s.  Sim 
One  was  controlled  by  a  hybrid  digital  and  analog  computer  and  had  many  features 
of  today’s  high-physical- fidelity  simulators:  Sim  One  could  breathe,  had  a  heartbeat, 
displayed  blood  pressure,  was  able  to  blink,  and  could  open  and  close  its  mouth.  In 
addition,  it  was  able  to  respond  to  four  intravenously  administered  drugs  and  two 
gases.  Sim  One  was  utilized  in  studies  of  proficiency  among  anesthesiology  resi¬ 
dents.  The  evaluation  of  the  effectiveness  of  this  simulator  indicated  that  simulator- 
trained  residents  reached  professional  levels  of  performance  in  fewer  elapsed  days 
and  in  a  smaller  number  of  trials  in  the  operating  room  than  those  who  did  not  par¬ 
ticipate  in  simulator  training.  One  obstacle  in  the  wide  use  of  Sim  One  was  its  pur¬ 
chase  and  maintenance  costs. 

Approximately  at  the  same  time,  the  cardiopulmonary  patient  simulator  Harvey 
was  introduced  at  the  University  of  Miami  (http://www.gcrme.med.miami.edu/ 
about_history_overview.php).  Today’s  version  of  Harvey  simulates  cardiac  and  lung 
disease,  including  blood  pressure,  breathing,  pulses,  heart  sounds,  and  murmurs. 

A  significant  step  toward  the  acceptance  of  medical  simulation  can  be  attributed 
to  the  work  of  Gaba  at  Stanford  and  of  Good  and  Grevenstein  at  the  University  of 
Florida  in  the  1980s.  Their  work  led  to  the  creation  of  commercial  medical  simulators 
for  use  in  medical  training.  Commercial  availability  made  it  easier  to  acquire  and 
operate  medical  simulators,  culminating  in  the  establishment  of  simulation  centers 
at  medical  schools  nationwide.  Today’s  popularity  of  simulation  for  medical  education 
is  illustrated  by  the  fact  that  the  number  of  annual  simulation-related  publications 
increased  more  than  tenfold  between  the  1990s  and  the  2000s  (Rosen,  2008).  Excellent 
reviews  of  the  history  of  medical  simulation  and  a  timeline  of  its  development  can  be 
found  in  Grenvik  and  Schaefer  (2004)  and  Rosen  (2008),  and  a  more  focused  discussion 
of  the  history  of  mannequin-based  simulators  can  be  found  in  Cooper  and  Taqueti 
(2004). 

When  comparing  the  development  of  simulation  in  aviation  and  in  medicine,  sev¬ 
eral  similarities  become  apparent,  leading  to  issues  we  will  discuss  in  depth: 

1.  Simulation  is  a  function  of  technology  available;  technological  advances 
allow  more  “realistic”  simulation.  However,  it  is  important  to  understand 
what  exactly  needs  to  be  realistic  in  successful  simulation  training. 
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2.  Technological  advances  lead  to  a  broadening  of  the  simulation  training 
goals,  exceeding  the  initial  basic  skills  training.  But  it  is  not  clear  how  to 
identify  critical  skills  and  how  to  develop  effective  simulation  training. 

3.  The  costs  of  simulation  technology  can  be  a  limiting  factor.  Thus,  it  is 
important  to  clearly  understand  the  economic  costs  and  benefits  of  simu¬ 
lation  use  in  training. 


HOWTO  DESCRIBE  SIMULATION? 

By  modern  standards,  early  aviation  simulators,  such  as  the  Link  Trainer,  appear 
crude  because  they  did  not  provide  visual  feedback  nor  did  they  use  accurate 
physical  modeling  of  aircraft  aerodynamics.  Nevertheless,  such  simulators  effectively 
supported  the  psychological  aspects  of  instrument-only  flying.  The  underlying 
assumption  for  pursuing  high  physical  realism  in  simulation  is  “ . . .  that  by  brute  engi¬ 
neering  force  all  physical  and  cognitive  aspects  of  a  task  can  be  captured”  (Caird, 
1996,  p.  127).  There  is  a  strong  predilection  toward  highly  realistic  simulation  (Scerbo 
&  Dawson,  2007;  Smallman  &  St.  John,  2005).  However,  this  preference  is  problematic 
because  physical  realism  alone  is  neither  necessary  nor  sufficient  for  effective  train¬ 
ing  (Adams,  1979;  Beaubien,  2004;  Kozlowski  &  DeShon,  2004;  Salas,  Bowers,  & 
Rhodenizer,  1998).  Instead,  it  is  vital  that  the  simulation  supports  the  psychological 
processes  required  for  a  specific  task  (Adams,  1979;  Kozlowski  &  DeShon,  2004).  For 
example,  instrument-only  flying  does  not  necessarily  require  an  exact  model  of 
physics  and  certainly  not  the  visual  simulation  of  weather  conditions. 

Typically,  the  term  fidelity  refers  to  physical  fidelity.  This  definition  is  indicative  of  the 
common  engineering  approach  to  simulation.  Simulator  characteristics  are  multidi¬ 
mensional  (related  but  partially  dissociable)  and  on  a  continuum.  We  use  high  and  low 
to  refer  to  the  admittedly  somewhat  subjective  and  ill-defmed  end  points  of  physical  and 
psychological  fidelity.  Other  characteristics  common  to  simulation  include  tractability, 
realism,  and  engagement.  We  use  realism  by  itself  to  refer  to  the  plausibility  of  events  and 
action  consequences  in  the  simulation  relative  to  the  real  world.  These  characteristics 
will  now  be  discussed. 

Simulator  Characteristics 

Fidelity.  Stimulus  fidelity  refers  to  an  exact  match  between  the  sensory  output  of  the 
simulator  and  the  actual  system  (Stoffregen,  Bardy,  Smart,  &  Pagulayan,  2003).  Thus,  a 
simulator  with  perfect  fidelity  would  be  indistinguishable  from  the  actual  system.  How¬ 
ever,  complete  physical  realism  and  environment  duplication  in  simulations  may  never 
be  possible  to  achieve  (Gibson,  1971;  Hochberg,  1986;  Stappers,  Gaver,  &  Overbeeke, 
2003;  Stoffregen  et  al.,  2003).  Even  though  complete  stimulus  fidelity  may  not  be  achiev¬ 
able,  the  question  is,  is  this  is  even  necessary?  For  example,  perceivers  likely  do  not  attend 
to  all  of  the  information  provided  to  the  same  extent  (Stoffregen  et  al.,  2003),  permitting 
some  degradation  of  fidelity.  One  method  to  achieve  high  levels  of  simulator  fidelity  is 
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by  use  of  model  approximations  and  alternate  dynamics  (Stappers,  1997, 1998;  Stappers 
&  Waller,  1993). 

An  important  consideration  is  the  level  of  physical  fidelity  needed  to  be  effective.  A 
simulator  used  for  training  should  have  a  level  of  fidelity  required  to  successfully  train 
the  individuals  using  the  simulator  (Salas  et  al.,  1998).  Therefore,  the  development  of 
simulation  models  requires  a  process  of  critical  feature  selection  and  quantitative 
description  of  relationships  to  accurately  mimic  the  actual  system.  An  effective  simula¬ 
tion  emulates  important  properties  and  maintains  critical  relationships  while  focusing 
on  the  purpose  of  the  simulation. 

In  general,  there  is  a  preference  for  high  visual  realism  by  users  and  designers 
(Smallman  &  St.  John,  2005).  The  conception  of  simulation  development  and  design 
solely  as  an  engineering  problem  and  less  as  a  knowledge  transfer  problem  may 
explain  the  common  inclination  toward  high  physical  realism.  There  are  two  potential 
disadvantages  to  high-fidelity  simulations:  cost  and  an  absence  of  training  benefits. 
High-fidelity  simulation  tools  are  generally  much  more  expensive  than  low-fidelity  sim¬ 
ulations  (Hopkins,  1974).  In  aviation,  a  high-physical-fidelity  simulation  does  not  neces¬ 
sarily  result  in  more  positive  knowledge  transfer  than  low-physical-fidelity  simulation  and 
sometimes  even  results  in  diminished  transfer  (Adams,  1979;  Caird,  1996;  Kozlowski  & 
DeShon,  2004).  The  allure  of  high-fidelity  simulation  parallels  a  phenomenon  in  the 
graphical  display  of  visual  information:  A  visual  display  with  more  realistic  graphics  is 
preferred  over  one  with  less  realistic  graphics,  even  when  the  less  realistic  display  leads  to 
a  better  understanding  of  the  information  (Smallman  &  St.  John,  2005).  Smallman  and 
St.  John  (2005)  call  this  phenomenon  “naive  realism.”  We  borrow  the  term  naive  realism 
to  refer  to  high-physical-fidelity  simulation  that  ignores  psychological  aspects  of  the 
training  task. 

Psychological  and  physical  fidelity.  Fidelity  extends  beyond  the  physical  realm. 
Fidelity  can  also  apply  to  subjective  experience.  Psychological  and  physical  fidelity  are 
partially  related  but  also  dissociable  and  encompass  the  two  major  dimensions  of  fidelity 
(see  Rehmann,  1995  for  subtypes  of  fidelity).  Physical  fidelity,  also  called  engineering 
fidelity  (Miller,  1957)  or  experiential  fidelity  (Stoffregen  et  al.,  2003),  is  the  degree  to 
which  the  physical  characteristics  of  the  system  are  replicated  within  the  simulation. 
Physical  fidelity  can  be  measured  by  the  degree  to  which  a  simulation  replicates  the  sen¬ 
sory  characteristics  of  the  actual  environment.  Psychological  fidelity,  also  called  action 
fidelity  (Stoffregen  et  al.,  2003),  functional  fidelity  (Moroney  &  Moroney,  1999),  or  task 
fidelity  (Rehmann,  1995),  is  the  degree  to  which  the  simulation  captures  the  psychologi¬ 
cal  aspects  of  the  real-world  task  or  activity.  Research  has  repeatedly  shown  that  psycho¬ 
logical  fidelity  is  fundamental  for  transfer  from  the  simulator  to  the  real-world  system 
(e.g.,  Kozlowski  &  DeShon,  2004). 

Learning  in  a  simulated  environment  does  not  necessarily  require  high  physical  fidel¬ 
ity.  Effective  training  can  often  be  accomplished  with  the  use  of  low-fidelity  simulators 
(Jentsch  &  Bowers,  1998;  Koonce  &  Bramble,  1998;  Patrick,  1992).  This  fact  does  not 
imply  that  high-fidelity  simulation  should  always  be  avoided;  instead,  maintaining  psy¬ 
chological  fidelity  to  the  task  (based  on  the  results  of  a  task  analysis,  see  Framework  for 
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Effective  Simulation  Training  section)  should  be  used  to  determine  the  appropriate  level 
of  physical  fidelity.  To  support  psychological  fidelity,  some  level  of  physical  fidelity  is 
necessary,  but  high  physical  realism  alone  is  not  sufficient.  For  example,  if  the  task  is 
suturing,  a  pig’s  foot  has  high  physical  fidelity  for  the  properties  of  skin,  a  relevant  aspect 
of  the  actual  task  (Barnes,  Lang,  &  Whiteside,  1989).  However,  a  pig’s  foot  would  not 
support  psychological  fidelity  for  providing  the  patient  wound  care  instructions.  An 
alternative  to  a  pig’s  foot  is  a  cadaver,  because  as  an  entire  human  body  it  has  greater 
physical  realism,  although  this  physical  realism  may  be  irrelevant  or  even  distracting  to 
the  task  of  suturing.  These  examples  demonstrate  that  psychological  fidelity  must  be 
supported  by  physical  realism  rather  than  general  high  fidelity. 

Another  example  is  simulating  tissue  handling  in  surgery.  This  task  requires  the 
knowledge,  skills,  and  abilities  to  deliver  the  appropriate  amount  of  force  for  cutting, 
pulling,  and  other  actions  (Al-Kadi  et  al.,  2012).  Training  on  virtual  surgical  simulators 
with  haptic  feedback  leads  to  better  instrument  control  and  accuracy  in  clinical  practice 
(Al-Kadi  et  ah,  2012;  Kim,  Rattner,  &  Srinivasan,  2003).  In  addition  to  haptic  feedback, 
adding  computer  animation  to  the  surgical  surfaces  also  improves  skill  acquisition 
because  it  is  relevant  to  the  perception  of  surface  elasticity  and  other  properties  that 
matter  for  surgery  (Basdogan  et  ah,  2004).  Although  the  fidelity  of  a  simulator  is  gener¬ 
ally  reported  in  terms  of  the  physical  resemblance  to  the  actual  simulated  system,  its 
psychological  fidelity  is  also  critical  to  the  successful  application  of  simulation. 

Tractability,  realism,  and  engagement.  A  tractable  simulation  is  one  that  can  be 
readily  used  for  its  intended  purpose.  For  example,  researchers  and  professionals  using 
simulators  for  training  purposes  are  often  caught  among  the  overwhelming  complexity 
of  the  field  and  the  lack  of  complexity  in  laboratory  contexts  (Brehmer  &  Dorner,  1993). 
Tractable  simulations  allow  the  researcher  to  manage  the  level  of  complexity  that  is  pre¬ 
sented  to  the  participant.  When  using  simulation  for  training,  tractability  refers  to  the 
amount  of  training  that  users  require  before  they  can  participate  in  the  simulation,  with 
training  negatively  correlated  with  tractability  (Gray,  2002). 

Realism  is  the  extent  to  which  actual  experiences  are  encountered  in  the  simulated 
system,  including  maintaining  the  functional  relationships  between  interacting  compo¬ 
nents  of  a  system.  For  example,  events  that  are  likely  to  happen  in  the  actual  system  will 
also  occur  in  the  simulated  system.  Realism  differs  from  fidelity  in  that  fidelity  focuses 
on  sensory  stimulation,  whereas  realism  focuses  on  plausible  events  and  the  conse¬ 
quences  of  actions  that  are  executed. 

Engagement  relates  to  a  participant’s  interest  in  and  motivation  toward  maintain¬ 
ing  high  levels  of  performance  throughout  the  scenario.  It  is  the  degree  to  which 
participants  take  the  simulation  seriously  (Gray,  2002).  Engagement  can  be  improved 
by  offering  financial  incentives  for  good  performance,  producing  an  interesting  or 
enjoyable  environment,  and  selecting  participants  with  an  interest  in  the  subject 
matter  (Ehret,  Gray,  &  Kirschenbaum,  2000;  Gray,  2002).  Physical  and  psychological 
fidelity  may  help  improve  engagement,  but  the  relationship  between  these  charac¬ 
teristics  is  not  well  understood  (see  Dieckmann,  Gaba,  &  Rail,  2007).  We  speculate 
that  irrelevant  physical  realism  (that  does  not  support  psychological  fidelity)  may 
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Figure  5. 1.  The  787  Simulator  Flight  Deck.  Copyright  ©  Boeing  (left).  Laerdal  Sim  Man  3G  high 
fidelity  patient  simulator  located  in  a  simulated  operating  room  at  the  Anesthesiology  Center 
for  Patient  Simulation  at  the  University  of  Utah  (right).  Copyright  2013  by  Noah  Syroid. 


increase  engagement  factors,  such  as  motivation,  yet  negatively  impact  psychological 
fidelity  by  being  potentially  distracting. 

There  are  deliberate  distinctions  in  terms  of  tractability,  realism,  and  engagement 
among  different  types  of  simulators.  Research  simulations  typically  require  high  levels 
of  tractability  and  engagement,  but  they  may  not  necessarily  need  to  have  high  levels  of 
physical  realism.  Training  simulations  may  require  high  levels  of  relevant  physical  real¬ 
ism  (in  support  of  psychological  fidelity)  to  allow  appropriate  learning  of  events,  but 
they  may  not  require  high  levels  of  tractability  (e.g.,  initially,  the  clinical  task  is  standard¬ 
ized)  or  engagement  (e.g.,  learners  may  be  intrinsically  motivated)  because  skills  can  still 
be  acquired  without  these. 

Types  of  Simulators 

Many  simulators  involve  either  complete  or  partial  physical  recreations  of  the  simulated 
system,  such  as  cockpits  in  flight  simulators  and  patient  simulators  to  medical  proce¬ 
dures  (see  Figure  5.1).  Some  simulations  deliberately  exclude  certain  features  of  the 
simulated  system  to  reduce  complexity,  whereas  other  simulations  allow  for  component 
interaction  that  would  be  impossible  in  the  real  world  (e.g.,  computer  games).  A  simula¬ 
tion’s  application  typically  motivates  its  design.  Because  there  are  many  diverse  applica¬ 
tions  for  simulators,  each  with  unique  requirements,  there  are  a  great  variety  of 
simulators.  Five  general  classes  of  simulators  can  be  identified:  high  fidelity,  low  fidelity, 
scaled  worlds,  microworlds,  and  virtual  environments  (VEs). 

High-fidelity  simulators.  High-fidelity  simulations  have  a  close  resemblance  to  the 
dynamics  and  appearance  of  the  system  that  is  being  simulated.  Driving  simulators,  such 
as  the  National  Advanced  Driving  Simulator  (NADS;  www.nads-sc.uiowa.edu),  have 
the  identical  controls,  displays,  and  feel  of  an  automobile  since  NADS  uses  a  vehicle  as 
part  of  the  simulator.  The  graphical  display  extends  to  360°  of  a  photo-realistic 
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representation  of  the  environment.  In  addition  to  realistically  recreating  the  physical 
environment,  many  high-fidelity  simulators  attempt  to  mimic  the  complexity  of  the  real 
world  (Gray,  2002),  such  as  replicating  the  behaviors  of  other  drivers  in  traffic.  High- 
fidelity  simulations  do  not  have  to  be  complex,  however.  Realistic  replications  of  simple 
systems  can  still  have  high  fidelity  without  high  complexity.  Also,  systems  do  not  have  to 
be  replicated  at  full  scale.  Subsystems  of  complex  systems  can  be  created  as  stand-alone 
simulations  (Gray,  2002).  An  example  of  a  high-fidelity  simulator  currently  used  in 
medical  simulation  centers  is  the  Human  Patient  Simulator  (see  Figure  5.1). 

Low-fidelity  simulators.  Low-fidelity  simulators  do  not  have  a  close  physical  resem¬ 
blance  to  the  simulated  system,  but  they  may  be  rated  high  on  psychological  fidelity.  Thus, 
it  is  possible  that  a  simulator  has  low  fidelity  and  still  can  maintain  effectiveness.  For  exam¬ 
ple,  numerous  studies  use  a  continuous-pursuit  tracking  task  to  simulate  driving  (e.g., 
Strayer  &  Johnston,  200 1 ) .  An  example  of  the  use  of  a  low- fidelity  simulator  in  health  care 
is  the  use  of  a  pig’s  foot  for  suturing  a  wound. 

Scaled  worlds  or  part-task  simulators.  An  alternate  approach  is  to  have  a  simulation 
that  can  help  the  learner  to  develop  automaticity  for  individual  parts  of  the  task.  Simula¬ 
tions  of  this  type  have  been  labeled  as  scaled  worlds  (Ehret  et  al.,  2000)  or  part-task 
simulators  (Perkins,  2007).  Examples  for  this  type  of  simulator  can  range  from  a  central 
vascular  catheter  (CVC)  insertion  simulator  to  practice  catheter  insertion,  to  peg  trans¬ 
fer  tasks  in  a  box  simulator  to  simulate  endoscopic/laparoscopic  environment  (Klein 
et  al.,  2008),  to  using  an  explanted  pig’s  heart  as  an  ex  vivo  part-task  simulator  for  car¬ 
diac  surgery  (Fann  et  al.,  2010).  A  part-task  simulator  provides  an  artificial  environment 
that  focuses  on  a  subset  of  the  component  relationships  that  are  found  in  complex  envi¬ 
ronments  rather  than  all  functional  interactions  (Gray,  2002).  In  essence,  part-task  sim¬ 
ulators  are  partial- fidelity  simulators,  which  leave  out  certain  characteristics  in  an  effort 
to  reduce  the  complexity  of  the  system  and  to  allow  the  users  to  gain  proficiency  in 
subtasks  without  overwhelming  them  with  information  (Stoffregen  et  al.,  2003).  Scaled 
worlds  are  effective  for  trainings  using  the  part-task  training  (PTT)  methods  discussed 
next. 

Reducing  the  complexity  of  a  learning  environment  is  often  advantageous  (Sweller, 
1994).  Research  demonstrated  that  limiting  fidelity  during  training  can  improve 
learning  and  task  performance  (Flach,  Ficcio,  McMillan,  &  Warren,  1986;  Salas  et  al., 
1998;  Sebrechts,  Lathan,  Clawson,  Miller,  &  Trepagnier,  2003;  Stappers  8c  Overbeeke, 
2003;  Taylor,  Lintern,  Koonce,  Kaiser,  &  Morrison,  1991).  Some  of  this  advantage  is 
achieved  through  the  removal  of  perceptual  clutter  in  the  simulator,  allowing  focused 
allocation  of  cognitive  resources  to  the  training  task.  The  learner  can  focus  on  the 
acquisition  of  a  task  component  without  information  overload  (Sweller,  1994).  Also, 
learners  may  become  overwhelmed  and  may  be  especially  susceptible  to  frustration 
while  practicing  with  complex,  whole-task  simulations  (Mattoon,  1994),  reducing 
their  motivation  to  expend  effort  on  the  task.  Thus,  part-task  simulators  allow  the 
partition  of  an  otherwise  difficult  task  into  smaller,  more  manageable  components, 
which  is  advantageous  when  learning  involves  complex  tasks,  such  as  those  taught  in 
health  care. 
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Microworlds.  Microworlds  or  synthetic  environments  (Gray,  2002)  are  computer 
simulations  that  mimic  the  component  relationships  found  in  complex  systems,  such  as 
factories,  forest  fires,  and  townships.  Microworlds  are  complex,  dynamic,  and  opaque. 
Complexity  refers  to  the  number  of  system  components  and  how  these  components 
interact  with  one  another.  The  component  relationships  of  microworlds  can  change 
during  the  simulation  to  correspond  with  actions  taken  by  the  user,  making  them 
dynamic.  The  opacity  of  a  microworld  indicates  that  internal  component  relationships 
are  not  visible  to  the  user  but  instead  must  be  inferred.  A  key  advantage  to  the  use  of 
microworlds  in  the  research  domain  is  that  they  maintain  good  ecological  validity  and 
also  provide  a  high  level  of  experimental  control  (Gray,  2002).  The  PC-based  Anesthesia 
Simulator  developed  by  Anesoft  is  an  example  of  a  microworld  (http://www.anesoft. 
com/products/anesthesia-simulator.aspx). 

VEs.  VEs  are  computer-generated  situations  with  which  an  individual  can  interact 
(Satava,  2006;  Stappers  et  al.,  2003).  They  usually  contain  three  important  characteris¬ 
tics:  a  computer-generated  visual  image,  high  interactivity,  and  presence,  providing  the 
user  with  the  feeling  of  being  immersed  in  the  simulated  environment  (Satava  &  Jones, 
2003).  Currently,  VEs  have  increasing  popularity  in  education  and  research.  They  have 
been  used  as  training  tools  in  many  areas,  such  as  spatial  navigation  (e.g.,  Sebrechts  et  al., 
2003)  and  teleoperation  (e.g.,  remote  control  of  robotics  in  dangerous  settings;  e.g., 
Stappers  et  al.,  2003).  They  have  been  used  to  enhance  minimally  invasive  surgical  tech¬ 
niques,  such  as  laparoscopic  surgery  and  endoscopies  (Haque  &  Srinivasan,  2006).  For 
example,  the  Virtual  Environments  for  Surgical  Training  and  Augmentation  Project  rep¬ 
resents  a  research  and  development  program  with  the  aim  of  improving  the  understand¬ 
ing,  assessment,  and  training  of  surgical  skills  using  VEs  for  training  (Tendick  et  al., 
2000).  Seymour  et  al.  (2002)  demonstrated  that  surgical  residents  whose  training  was 
complemented  with  the  use  of  a  virtual  reality  surgical  simulation  made  six  times  fewer 
errors  when  performing  a  laparoscopic  cholecystectomy.  Additionally,  residents  who 
were  trained  with  standard  training  methods  were  nine  times  more  likely  to  fail  to  prog¬ 
ress  and  five  times  more  likely  to  injure  the  patient. 

For  VE  research,  presence  and  immersion  are  commonly  used  to  describe  the  level  of 
psychological  fidelity  created  by  physical  fidelity  (Witmer  &  Singer,  1998).  Presence 
refers  to  a  feeling  of  “being  there,”  a  suspension  of  disbelief  of  being  in  the  simulation 
(Witmer  &  Singer,  1998),  whereas  immersion  is  the  psychological  state  of  being 
enveloped  or  absorbed  by  the  simulation.  An  example  of  a  VE  in  health  care  used  for 
medical  education  and  team  training  is  3DiTeams  developed  by  Duke  University 
Medical  Center  and  Virtual  Heroes,  Inc.  Other  examples  of  VEs  are  Trauma  Connect 
and  Virtual  Medical  Simulation  Training  Center,  both  for  training  combat  medics 
(Sotomayor,  Salava,  &  York,  2012). 

Alternative  Classifications 

A  broader  schema  of  describing  the  diversity  of  simulations  being  used  in  health  care 
was  developed  by  Gaba  (2004),  identifying  11  dimensions  to  categorize  simulation 
applications.  The  dimensions  Gaba  (2004,  p.  i4)  lists  are:  the  purpose  and  aims  of 
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simulation,  the  unit  of  participation  in  the  simulation,  the  experience  level  of  simulation 
participants,  the  health  care  domain  to  which  simulation  is  being  applied,  the  health 
care  discipline  of  participating  health  care  personal,  the  types  of  knowledge,  skill,  atti¬ 
tudes  and  behaviors  addressed  in  the  simulation,  the  age  of  the  simulated  patient,  the 
technology  applicable  for  simulation,  the  site  of  the  simulation  participation,  the  extent 
of  direct  participation  in  the  simulation,  and  the  feedback  method  accompanying  the 
simulation.  Gaba  (2004)  also  discusses  two  of  the  oldest  simulations  used  in  health  care: 
verbal  role-playing  and  simulated  standardized  patients  played  by  actors.  One  of  the 
major  benefits  of  this  comprehensive  approach  to  classify  simulation  applications  is  that 
it  provides  a  standard  to  describe  simulations.  Finally,  a  conceptually  complementary 
perspective  on  simulation  is  provided  by  Dieckmann,  Gaba,  and  Rail  (2007),  who  ana¬ 
lyze  and  describe  simulation  as  social  practice  that  involves  not  only  the  simulator  and 
other  technical  means  but  also  other  humans. 


WHY  SIMULATION  IN  HEALTH  CARE? 

In  health  care,  inadequate  training  is  a  major  contributor  to  high  rates  of  error,  large 
numbers  of  preventable  adverse  events,  and  many  patient  deaths  (Kohn,  Corrigan, 
&  Donaldson,  1999;  Leape,  1994;  Rodriguez-Paz  et  al.,  2009).  Health  care  lags  behind 
other  safety  critical  industries,  notably  aviation  (Berwick  &  Leape,  1999;  Durso  & 
Drews,  2010).  Simulation  training  has  enormous  potential  to  improve  patient  safety 
by  providing  a  safe  training  environment  for  repeated  practice,  exposure  to  rare, 
complex  conditions  and  events,  and  assessment  and  immediate  feedback  on  clinical 
performance  (e.g.,  adherence  to  best  practices,  local  practice  protocols,  and  other 
measures  of  patient  safety)  (Gaba,  2004;  Grenvik,  Schaefer,  DeVita,  &  Rogers,  2004; 
Rodriguez-Paz  et  ah,  2009;  Salas,  Wilson,  Burke,  &  Priest,  2005). 

Traditional  training  is  costly  in  terms  of  patient  safety,  time,  efficiency,  and  mone¬ 
tary  expense  (Haluck  &  Rrummel,  2000;  Kneebone,  2010)  and  in  some  cases  not 
highly  effective  to  reinforce  knowledge  (Steadman  et  ah,  2006).  In  addition,  due  to  ethi¬ 
cal  and  other  practical  reasons,  patients  cannot  be  used  in  many  aspects  of  health  care 
training  (Sebrechts  et  al.,  2003).  Traditional  alternate  forms  of  training,  such  as  the  use 
of  animals  and  cadavers,  are  often  limited  in  their  availability.  Even  when  these  alternates 
are  available,  individual  variability  and  specimen  diversity  may  not  allow  the  training  to 
be  generalized  (Issenberg  et  al.,  1999;  Sebrechts  et  ah,  2003).  Therefore,  interactive  simu¬ 
lations  provide  alternatives  to  traditional  training  because  they  replicate  the  actual  clini¬ 
cal  task  and  activities  using  training  situations  without  the  real-world  risk  to  human 
health  and  life. 

The  use  of  simulation  in  health  care  has  recently  grown  (Haluck  &  Rrummel,  2000; 
Haque  &  Srinivasan,  2006)  and  gained  wider  acceptance  (Issenberg  et  ah,  1999). 
During  the  past  several  decades,  technological  advancements  have  had  a  great 
impact  on  medical  diagnosis  and  treatment  processes;  however,  there  is  a  discon¬ 
nect  between  these  advances  and  current  training  practice  (Haluck  &  Rrummel, 
2000).  Simulation  in  health  care  has  significant  potential  as  a  training  and  educational 
tool  (Satava  &  Jones,  2003);  however,  as  with  any  training  technique,  it  requires  critical 
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assessment  to  ensure  effectiveness.  Evaluating  the  effectiveness  of  simulators  used  for 
training  comes  with  additional  limitations.  These  limitations  include  identifying  an 
adequate  number  of  patients  with  the  target  condition  or  procedure  to  use  as  a  control 
group,  ethical  concerns  from  involving  patients  at  all  as  a  control  group,  and  small  sam¬ 
ple  sizes  due  to  a  limited  number  of  trainees  (Ost  et  al.,  2001). 

Aviation  and  Health  Care:  Similarities  and  Differences 

To  increase  the  understanding  of  the  role  of  simulation  in  health  care,  it  is  helpful  to 
compare  it  with  another  domain  in  which  simulation  has  been  successfully  used:  avia¬ 
tion.  By  better  understanding  the  similarities  and  differences  between  these  domains,  it 
is  possible  to  identify  when  simulation  training  approaches  can  be  adopted  without 
modification  and  when  a  modification  is  needed. 

Health  care  shares  commonalities  with  aviation,  but  there  are  key  differences  between 
the  two  domains  (Durso  &  Drews,  2010).  In  Table  5. 1,  we  outline  the  differences  between 
aviation  and  health  care  systems  in  terms  of  the  social,  natural,  and  technical  systems 
and  illustrate  some  of  the  limitations  associated  with  simulation  use. 

Table  5.1  illustrates  important  differences  to  take  into  account  when  applying  simula¬ 
tor  training  approaches  from  aviation  to  health  care.  Among  the  most  important  differ¬ 
ences  are  those  that  involve  issues  of  standardization  of  equipment  and  activities,  the 
level  of  specialization  involved,  high  inter-individual  variability  in  patients,  and  differ¬ 
ences  in  safety  culture.  Without  taking  into  account  these  differences,  simulation  train¬ 
ing  in  health  care  runs  the  risk  of  failure  or  at  least  the  risk  of  being  not  as  effective  as 
possible.  Thorough  task  analysis  before  initiating  simulator  training  development  can 
help  avoid  this  risk. 

Simulation  in  Health  Care 

Medical  training.  Many  simulators  used  in  health  care  have  been  demonstrated  to  be 
valuable  and  cost  effective  tools.  One  such  simulator  is  the  Harvey  (Issenberg  et  al.,  1999; 
see  also  Simulation  in  Health  Care  section),  a  life-sized  mannequin  interface  simulating 
up  to  27  cardiac  conditions.  Harvey  allows  novice  students  to  learn  basic  procedures, 
such  as  recognition  of  heart  murmurs,  whereas  advanced  students  learn  how  to  correlate 
heart  sounds  with  respiration.  One  of  Harvey’s  most  impressive  attributes  is  the  amount 
of  testing  performed  to  verify  its  training  effectiveness.  A  multicenter  study  incorporat¬ 
ing  208  senior  medical  students  from  five  medical  schools  showed  that  4th-year  students 
who  used  the  Harvey  simulator  as  part  of  their  cardiology  elective  had  better  post-test 
performance  on  a  cardiology  patient  simulator  than  the  traditionally  trained  control 
group  (Woolliscroft,  Calhoun,  Tenhaken,  &  Judge,  1987).  In  addition,  the  study  addressed 
concerns  that  simulator  training  would  negatively  affect  physician-patient  interaction 
because  no  differences  were  found  in  how  patients  perceived  the  professional  behavior 
of  the  students.  A  second  study  of  203  second-year  medical  students  demonstrated  that 
the  use  of  the  Harvey  simulator  in  conjunction  with  a  physical  skills  course  resulted  in 
significant  improvements  in  cardiac  examination  skills  (Woolliscroft,  Calhoun,  Ten¬ 
haken  &  Judge,  1987).  A  survey  completed  by  five  medical  schools  using  the  Harvey 
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Table  5.1.  Aviation  and  Health  Care  System  Differences 


Domain 


Aviation 


Health  Care 


Equipment 


Activities  and  personnel 


Interactions  between 
systems 


Standardized:  Training  and 
licensure  are  aircraft  specific. 


Standardized:  Checklists  and 
protocols;  strong  adherence. 


Generalized:  Copilot  can 
perform  the  same  activities  as 
the  pilot. 

Technical:  Aircraft  are  consistent 
operational  environments; 
some  nontechnical  systems 
(human  communication  and 
weather)  also  present. 

Culture:  Reduced  hierarchy. 
Individuals  at  different  levels 
can  directly  communicate 
(e.g.,  flight  attendant  or  copilot 
raising  a  concern  with  the 
pilot).  Safety  culture  with 
anonymous  reporting  of  near 
misses;  proactive. 


Lack  of  standardization:  Wide 
variations  in  equipment; 
training  and  licensure  are 
rarely  equipment  specific. 

Lack  of  standardization:  Huge 
variations  in  local  practices; 
weak  adherence  to  best 
practices. 

Highly  specialized:  Minimal 
duplication  of  activities  across 
roles. 

Socio-natural:  Patients  are 
highly  variable;  operational 
environment  is  inconsistent 
and  often  suboptimal; 
some  technical  systems 
(equipment). 

Culture:  Rigid  hierarchy.  Minimal 
safety  culture  with  few 
reported  close  calls;  reactive. 


simulator  as  part  of  their  curriculum  (Issenberg  et  al.,  1999),  reported  high  weekly  simu¬ 
lator  usage  (22  hr)  with  the  majority  (17  hr)  spent  by  small  groups  of  students  in  self¬ 
learning  mode.  The  remaining  time  involved  instructors  teaching. 

Traditional  health  care  training  methods  involve  high  monetary  costs  and  large  time 
commitments  from  students  and  instructors.  The  inclusion  of  simulators  in  the  curricu¬ 
lum  has  the  potential  to  provide  student  training  comparable  to  traditional  training 
while  freeing  up  instructors.  In  addition,  traditional  training  is  dependent  upon  the  con¬ 
dition  or  disease  of  the  patient  population.  Time  costs  are  involved  while  waiting  for  a 
patient  on  which  to  learn  a  procedure.  Bridges  and  Diamond  (1999)  report  that  in  the 
United  States,  nonsupply  financial  costs  related  to  operating  room  (OR)  training  for 
general  surgery  residents  alone  is  over  $53  million  annually.  When  considering  other 
trainees  (e.g.,  anesthesiologists,  surgical  trainees,  nurses)  within  the  OR  and  in  other 
health  care  domains,  the  national  training  cost  is  enormous  (Haluck  &  Krummel,  2000). 

Nursing.  In  addition  to  the  use  of  simulators  to  train  physicians,  simulators  are  also 
being  used  successfully  to  train  other  health  care  professionals,  such  as  nurses.  There  is  a 
wide  range  of  simulators  that  are  being  used,  ranging  from  high  fidelity  human  patient 
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simulation  mannequins  (HPSMs)  to  task  trainers  (e.g.,  an  arm  for  intravenous  line 
insertion  training).  Simulation  is  being  used  in  areas  including  fundamentals  of  nursing, 
medical,  surgical,  and  pediatric  care  and  high-acuity  nursing.  A  recent  survey  of  1,060 
pre-licensure  nursing  programs  in  the  United  States  documented  the  widespread  use  of 
simulation  in  nursing  education  (Hayden,  2010).  Respondents  to  the  survey  emphasized 
the  importance  of  simulation  in  the  nursing  curriculum;  77%  said  they  do  or  would 
substitute  clinical  time  with  simulation  time  if  permitted  by  regulations.  Given  the  pop¬ 
ularity  of  simulation  in  nursing,  it  is  important  to  briefly  review  the  literature  on  the 
impact  of  simulation  training  on  performance. 

There  is  accumulating  evidence  that  the  use  of  simulation  in  nursing  has  a  significant 
impact  on  several  measures  of  performance.  For  example,  Lapkin,  Levett-Jones,  Bell- 
chambers,  and  Fernandez  (2010)  report  in  a  systematic  literature  review  that  un¬ 
dergraduate  nursing  students  improve  in  knowledge  acquisition  and  critical  thinking 
skills  from  the  use  of  HPSMs.  Similarly,  Cant  and  Cooper  (2010)  report  that  simulation 
in  nursing  training  increases  knowledge,  critical  thinking  ability,  satisfaction,  and  confi¬ 
dence  compared  to  control  groups.  Finally,  Pauly-O’Neill  (2009)  demonstrated  signi¬ 
ficant  improvement  in  pediatric  medication  administration  performance  among 
simulation  users  compared  to  a  control  group. 

Effectiveness  of  simulation  training.  Overall,  researchers  analyzing  the  effectiveness 
of  health  care  simulators  suggest  that  incorporating  effective  simulation  into  a  tradi¬ 
tional  medical  education  curriculum  will  produce  a  positive  outcome  on  student  train¬ 
ing.  There  is  growing  evidence  that  simulations  can  be  useful  in  helping  experienced 
health  care  professionals  maintain  and  evaluate  their  technical  skills  (Haluck  &  Krum- 
mel,  2000;  Issenberg  et  al.,  1999).  Simulators  are  currently  used  to  teach  procedures 
including  neurosurgery,  orthopedic  surgery,  plastic  surgery,  coronary  artery  bypass  sur¬ 
gery,  intraocular  surgery,  endoscopy  and  laparoscopic  surgery,  fluoroscopy,  cardiovascu¬ 
lar  and  pulmonary  disease  diagnosis  and  treatment,  anesthesia,  patient-centered 
programs,  basic  vital  signs  emergency  care,  infant  care,  neonatal  care,  obstetrics,  phle¬ 
botomy  and  catheter  placement,  biological  and  chemical  agent  exposure  diagnosis  and 
treatment,  HIV  treatment,  and  more  (Haque  &  Srinivasan,  2006;  Issenberg  et  al.,  1999). 

In  addition  to  training  of  technical  skills,  simulation  training  can  also  focus  on  the 
development  of  nontechnical  skills.  Examples  of  nontechnical  skills  are  interpersonal 
skills,  such  as  communication  (Baker,  Gustafson,  Beaubien,  Salas,  &  Barach,  2005), 
teamwork,  and  leadership,  and  cognitive  skills,  such  as  task  management,  situation 
awareness  (Gaba,  Howard,  &  Small,  1995),  and  decision  making  (see  Cosby  & 
Croskerry,  2003;  Gaba,  1992).  In  recent  years,  there  has  been  an  emergence  of  nontech¬ 
nical  skill  training  especially  in  anesthesia  and  surgery  (Fletcher  et  al.,  2003;  Gaba,  1989; 
Helmreich,  Wilhelm,  Klinect,  &  Merritt,  2001;  Yee  et  al.,  2005),  and  other  medical  spe¬ 
cialties  are  following  suit.  Rail  and  Gaba  (2005)  point  out  the  similarity  of  nontechnical 
skills  required  in  anesthesia  and  those  required  in  other  areas  of  health  care  (e.g.,  the 
intensive  care  unit).  Fletcher  et  al.  (2003)  provide  an  example  on  how  to  identify  impor¬ 
tant  nontechnical  skills  for  anesthesiologists.  The  Anesthetist’s  Non-Technical  Skills 
(ANTS)  behavioral  marker  system  includes  four  core  nontechnical  skills:  task 
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management,  team  working,  situation  awareness,  and  decision  making.  Flin  and  Maran 
(2004)  describe  a  simulator  based  course  that  targets  these  nontechnical  skills. 

Human  factors-based  work  focusing  on  nontechnical  skills  involves  different 
methods  to  identify  required  skills.  Among  those  methods  are  simulator-based  or 
real-life  observational  studies  (Donchin  et  al.,  1995),  studies  of  cognition  (Patel, 
Arocha,  &  Kaufman,  2001),  survey-based  methods,  and  root  cause  analyses. 
Ultimately,  the  choice  of  focusing  on  technical  or  nontechnical  skills  in  simulator 
training  is  a  function  of  the  training  goals  (see  Framework  for  Effective  Simulation 
Training  section).  In  recognition  of  the  potential  associated  with  the  use  of  simula¬ 
tion,  many  institutions  have  implemented  simulation  into  training  programs  and  cre¬ 
ated  simulation  centers  (Table  5.5  lists  some  examples). 

Diagnostic  and  preoperative  simulation.  Simulation  has  great  potential  in  the  area 
of  diagnosis.  The  acquisition  of  accurate  data  provides  the  basis  for  correct  medical 
diagnoses  (Satava  &  Jones,  2003).  Data  about  a  patient  are  obtained  through  medical 
history,  laboratory  tests,  physical  examination,  and  increasingly  with  the  aid  of  diag¬ 
nostic  imaging  techniques.  Three-dimensional  “simulated”  representations  of  the 
human  body  can  be  created  with  the  use  of  computed  tomography  (CT),  magnetic 
resonance  imaging  (MRI),  functional  MRI  (fMRI),  positron  electron  tomography 
(PET),  ultrasound,  and  single  photon  computed  tomography  (SPECT).  Images  col¬ 
lected  for  diagnostic  purposes  can  be  used  to  preoperatively  plan  procedures.  The 
patient-specific  images  can  be  imported  into  a  simulator  to  create  a  representation 
customized  for  that  patient.  This  type  of  tailored  simulation  allows  physicians  to  prac¬ 
tice  procedures  with  a  patient-specific  representation  before  executing  them  on  the 
patient  (reflecting  also  one  of  the  differences  between  aviation  and  health  care  simula¬ 
tion).  Applications  that  use  such  techniques  include  plastic  surgery  (J.  Rosen,  1992), 
bony  face  deformities  (Gleason  et  al.,  1995),  and  tendon  transplant  in  the  leg  (Delph 
&  Zajac,  1992). 


THE  DEVELOPMENT  OF  EFFECTIVE  SIMULATION  TRAINING 

For  simulation  training  to  be  effective,  it  must  support  the  key  psychological  aspects 
of  the  real-world  task.  In  the  Framework  for  Effective  Simulation  Training  section,  we 
propose  a  framework  for  understanding  the  psychological  fidelity  of  a  clinical  task 
(e.g.,  scope  of  the  task,  the  task  steps,  performance  measures,  and  training  goals) 
and  using  this  information  to  choose  the  simulation  tool.  The  emphasis  on  psycho¬ 
logical  fidelity  is  critical  to  the  ultimate  goal  of  all  training:  that  the  knowledge,  skills, 
and  abilities  acquired  during  training  transfer  to  improve  real-world  performance 
(Auffrey,  Mirabella,  Siebold,  &  Schaab,  2001;  Baldwin  &  Ford,  1988;  Roscoe  &  Williges, 
1980;  Salas,  Tannenbaum,  Kraiger,  8c  Smith-Jentsch,  2012).  Although  the  proposed 
framework  does  not  guarantee  transfer  of  training,  it  is  based  on  the  current  empiri¬ 
cal  and  theoretical  evidence  for  developing  effective  training.  In  the  Transfer  of  Training 
section,  we  discuss  measuring  the  transfer  of  training. 
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In  health  care,  there  is  growing  evidence  that  simulation  training  improves  clinical 
performance;  however,  there  are  limited  data  that  this  improved  performance  results 
in  better  clinical  outcomes  for  patients  (see  the  literature  reviews  and  meta-analyses 
by  Cook,  Hatala,  &  Brydges,  2011;  McGaghie,  Issenberg,  Cohen,  Barsuk,  &  Wayne, 
2011;  McGaghie,  Issenberg,  Petrusa,  &  Scalese,  2010;  Ross,  Kodate,  Anderson,  Thomas, 
&  Jaye,  2012;  Schout,  Hendrikx,  Scheele,  Bemelmans,  &  Scherpbier,  2010). 

The  Institute  for  Health  Care  Improvement  (n.d.)  recommends  simulation  training 
for  high-risk  situations  where  there  is  empirical  evidence  for  its  effectiveness.  Empirical 
evidence  for  the  effectiveness  of  clinical  decision  support  systems  is  strikingly  similar, 
with  moderate  evidence  of  improved  clinical  performance  but  limited  data  on  patient 
outcomes  (Garg  et  al.,  2005).  The  general  measurement  challenges  in  health  care  are  cov¬ 
ered  in  detail  in  the  Quality  of  Care  and  Clinical  Performance  section. 

The  lack  of  widespread  data  on  training  effectiveness  translating  into  effective  real- 
world  clinical  performance  is  not  unique  to  health  care  training.  Measuring  transfer  of 
training  is  frequently  overlooked  across  all  types  of  industries  (Salas  et  al.,  2012). 
Generally,  assessment  of  performance  in  clinical  practice  and  clinical  outcomes  after 
simulation  training  are  uncommon  (Lynagh,  Burton,  &  Sanson-Fisher,  2007;  Teteris, 
Fraser,  Wright,  &  McFaughlin,  2012).  Evaluating  and  comparing  actual  clinical  perfor¬ 
mance  across  training  methods  in  health  care  is  especially  challenging  because  of  the 
variety  of  potential  methodological  and  measurement  confounds  (see  sections  Transfer 
of  Simulation  Training  to  Clinical  Practice  and  Quality  of  Care  and  Clinical  Performance) . 
However,  simulation-based  training  was  widely  implemented  in  other  safety  critical 
industries  (e.g.,  aviation,  military)  prior  to  extensive  empirical  evidence  for  transfer  to 
real-world  performance  (Alessi,  2000). 

Assessment  of  clinical  performance,  whether  in  a  simulator  or  ideally  in  actual  prac¬ 
tice,  is  an  essential  component  of  simulation  training  (Issenberg  et  al.,  1999;  Issenberg  & 
Scalese,  2008).  Another  aspect  of  assessment  is  to  provide  immediate  feedback  to  trainees 
(Grenvik  et  al.,  2004).  Using  simulation,  trainees  can  be  systematically  evaluated  with 
performance  measures  or  indicators  of  competence  (Cooper  &  Taqueti,  2008;  Epstein, 
2002;  Good,  2003)  for  both  preclinical  (Fitch,  2007)  and  clinical  training  (Okuda  et  al., 
2009).  For  clinical  training,  simulation  performance  has  started  to  become  part  of  the 
criteria  for  board  certification  and  licensure  in  medicine  (Dillon,  2004;  Okuda  et  al., 
2009),  nursing  (Nehring,  2008),  and  dentistry  (Holmboe,  Rizzolo,  Sachdeva,  Rosenberg, 
&  Ziv,  20 1 1 ) .  A  common  method  for  assessing  student  and  trainee  clinical  competency  is 
the  Objective  Structured  Clinical  Exam,  which  is  a  checklist  assessment  of  key  clinical 
skills  administered  to  standardized  or  real  patients  (e.g.,  Harden,  Stevenson,  Wilson 
Downie,  &  Wilson,  1975). 

A  challenge  to  the  effectiveness  of  simulation,  whether  for  licensure  or  other  purposes, 
is  the  use  of  one-shot  testing  (e.g.,  Chambers,  Dugoni,  &  Paisley,  2004).  Similar  to  the 
lack  of  widespread  assessment  of  transfer  of  training,  this  issue  is  unique  neither  to  either 
health  care  nor  to  simulation.  There  are  considerable  limitations  with  using  single  item 
measures,  such  as  one  simulation  session,  to  assess  clinical  performance.  Psychometric 
properties  (i.e.,  measurement  validity  and  reliability;  e.g.,  Cronbach  &  Meehl,  1955)  of 
nonrepeated,  single-item  measures  may  be  inadequate,  although  it  depends  on  the  prop¬ 
erties  of  the  specific  measure  (Carmines  &  Zeller,  1979).  Moreover,  measurement  of 
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clinical  competency  (fitness  to  practice)  with  the  use  of  simulation  and/or  exams  has  the 
same  psychometric  challenges  (Wass,  Van  der  Vleuten,  Shatzer,  &  Jones,  2001),  specifi¬ 
cally  whether  the  measures  generalize  or  transfer  to  actual  clinical  practice. 

Framework  for  Effective  Simulation  Training 

Effective  simulation  training  requires  replicating  the  psychological  fidelity  of  the 
real-world  task  (Auffrey  et  al.,  2001;  Baldwin  &  Ford,  1988;  Kozlowski  &  DeShon, 
2004).  Consequently,  an  understanding  of  the  scope,  psychological  operations,  per¬ 
formance  measures,  and  training  goals  for  the  clinical  task  is  needed  prior  to  select¬ 
ing  the  simulation  tool.  If  the  simulation  tool  or  method  is  selected  first,  training  may 
fit  the  technological  capabilities  but  may  not  necessarily  support  the  psychological 
fidelity  of  the  clinical  task. 

To  develop  effective  simulation  training,  we  propose  the  following  framework 
(adapted  from  Auffrey  et  ah,  2001;  Baldwin  &  Ford,  1988;  Gaba,  2004;  Salas  et  ah,  2008; 
Salas  &  Burke,  2002)  to  guide  simulation  training  development: 

1.  Procedure/task  analysis:  What  is  the  procedure,  the  steps  to  successfully 
perform  the  task,  and  potential  incidents  or  adverse  events? 

a.  Scope  of  the  task:  Does  simulation  training  cover  parts  of  the  task  or  the 
entire  task?  Is  the  task  limited  to  technical  and  procedural  skills  (e.g.,  plac¬ 
ing  sutures),  diagnostic  skills  (e.g.,  selecting  the  appropriate  sutures  given 
the  laceration,  checking  for  the  presence  of  foreign  bodies),  patient  inter¬ 
action  (e.g.,  assessment  of  infection  risk  based  on  the  cause  of  the  lacera¬ 
tion  and  patient’s  medical  history,  assessment  of  drug  allergies),  or  all  of 
the  above  (adpated  from  Hays,  2010)?  Simulation  training  often  focuses 
on  tasks  involving  technical  and  procedural  skills  but  need  not  and  should 
not  be  limited  to  these  (Gaba,  2004). 

b.  Cognitive  task  analysis  (CTA):  What  methods  are  necessary  to  understand 
task  steps  and  sub-steps  in  terms  of  the  “. . .  knowledge,  thought  processes, 
and  goal  structures  that  underlie  task  performance”  (Chipman,  Schraagen, 

&  Shalin,  2000,  p.  3)?  Numerous  methods  for  conducting  CTA,  such  as 
observation,  structured  and  unstructured  interviews,  expert  opinion,  and 
instructions,  exist  (see  Cooke,  1994).  Applied  CTA  (ACTA)  provides  in¬ 
structions  for  a  practical  set  of  methods  (Militello  &  Hutton,  1998);  Chip- 
man  et  al.  (2000)  provide  an  introduction  to  general  approaches  to  CTA. 
Overall,  the  CTA  should  incorporate  the  following: 

i.  Evidence-based  practices:  Best  practices  for  the  most  effective  and  safe 
delivery  of  treatment  and  care  (Grol  &  Grimshaw,  2003;  Makic,  Von- 
Rueden,  Rauen,  &  Chadwick,  2011;  Timmermans  &  Berg,  2003). 

ii.  Focal  protocols:  Institution-specific  instructions/guidelines  that  may 
include  best  practices  and  equipment  manufacturer  recommendations 
(e.g.,  Timmermans  &  Berg,  2003). 
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iii.  Incidents  and  adverse  events:  A  series  of  events  or  factors  that  could 
result  in  a  patient  hazard  (Cook  &  Woods,  1994),  incidents  increase  the 
likelihood  of  an  adverse  event  (an  injury  due  to  delivery  or  failure  in 
the  delivery  of  treatment;  Rothschild  et  al.,  2005).  Adverse  events  can 
be  nonpreventable  or  preventable.  Preventable  incidents  and  adverse 
events  rarely  occur  due  to  a  single  cause  or  error  (see  Cook  &  Woods, 
1994;  DeLucia,  Ott,  &  Palmieri,  2009;  Drews,  2011;  Morrow,  North,  & 
Wickens,  2005;  Reason,  1995),  and  the  likelihood  of  an  adverse  event 
increases  following  suboptimal  clinical  performance  and  error.  How¬ 
ever,  incidents  and  adverse  events  may  also  arise  from  other  risk  factors: 
equipment  failure,  lack  of  staffing,  stress,  and  so  on  (also  see  error-pro¬ 
ducing  conditions).  Incorporating  incidents  and  adverse  events  into 
simulator  training  promotes  the  transfer  of  knowledge  and  skills  neces¬ 
sary  to  manage  their  occurrence  (DeAnda  &  Gaba,  1990;  Gaba,  How¬ 
ard,  Fish,  Smith,  &  Sowb,  2001;  Grenvik  et  al.,  2004).  An  example  of  an 
incident  is  a  faulty  ventilator  circuit  (Johnson,  Syroid,  &  Drews,  2008), 
which  becomes  an  adverse  event  if  not  detected  and  the  patient  stops 
breathing. 

iv.  Error-producing  conditions  (EPCs):  Characteristics  of  the  actual 
operational  environment  (e.g.,  multitasking,  insufficient  staff,  mental 
and  physical  task  demands,  fatigue,  and  time  pressure)  that  increase 
the  likelihood  of  erroneous  behavior  leading  to  an  increased  risk  of 
incidents  and  adverse  events  (Drews,  2011;  Drews,  Musters,  &  Samore, 
2008).  Like  incidents,  EPCs  can  be  mimicked  in  simulation  training 
and  may  help  better  prepare  clinicians  for  similar  conditions  in  the  real 
world. 

v.  Equipment:  Equipment  used  in  the  simulation  should  be  identical  to 
equipment  used  in  the  real  world  to  facilitate  matching  the  psychologi¬ 
cal  aspects  of  training  to  the  clinical  task  (Kozlowski  &  DeShon,  2004; 
Maran  &  Glavin,  2003).  Thus,  physical  fidelity  can  be  used  to  positively 
affect  psychological  fidelity. 

2.  Identify  the  training  goals  and  performance  measures 

a.  Training  goals  involve  the  simulation  training  objectives,  that  is,  key 
knowledge,  skills,  and  abilities.  For  the  example  of  placing  sutures,  train¬ 
ing  goals  are  (adapted  from  Hays,  2010)  assessment  and  minimization  of 
infection  risk,  selection  of  appropriate  sutures  for  the  wound,  and  proper 
technical  and  procedural  skills. 

Specific  training  goals  can  be  informed  by  elements  in  the  scope  of  the  activity  and 
CTA.  Also,  goals  depend  upon  the  purpose  of  training,  for  example,  preclinical,  certifica¬ 
tion/licensure,  or  a  continuing  education  for  experienced  clinicians  (Dillon,  2004; 
Nehring,  2008;  Okuda  et  al.,  2009).  One  method  for  achievement  of  training  goals  is 
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meeting  or  exceeding  a  criterion-based  level  of  performance  in  the  simulator  before 
advancing  to  the  same  task  in  the  real  world  (Roscoe,  1991). 

b.  Clinical  perfonnance  measures  involve  metrics  describing  clinical  per¬ 
formance,  for  example,  adherence  to  evidence-based  practices  and  local 
protocols.  Applied  to  the  example  of  placing  sutures,  the  aforementioned 
training  goals  can  be  operationalized  as  measures  (adapted  from  Hays, 
2010): 

i.  Assessment  and  minimization  of  infection  risk:  Take  the  patient’s 
medical  history,  assess  the  cause  and  severity  of  wound,  check  for 
presence  of  foreign  bodies,  and  treat  patient  with  antibiotics  if 
applicable. 

ii.  Selection  of  appropriate  sutures:  Assess  wound  severity  (depth, 
width,  and  type  of  wound)  and  select  sutures,  including  gathering 
required  supplies. 

iii.  Proper  technical  and  procedural  skills:  Aseptic  technique  (hand 
hygiene,  gloves,  sterile  equipment,  no  contamination  of  sterile 
equipment  or  gloved  hands  from  a  nonsterile  surface)  and  suturing 
technique  (spacing). 

iv.  Error:  Performance  can  be  assessed  in  the  context  of  simulated  inci¬ 
dents,  adverse  events,  and  EPCs.  If  possible,  performance  measure¬ 
ments  should  not  be  limited  to  assessment  during  training  but  also 
should  be  measured  in  clinical  practice.  Examples  include  failure  to 
ask  a  patient  about  drug  allergies  or  administering  the  wrong  type/ 
dose  of  a  medication. 


c.  Subjective  measures,  like  focus  groups,  interviews,  opinions  of  subject 
matter  experts,  and  surveys,  lack  a  “ground  or  an  objective  truth.”  Sub¬ 
jective  metrics  can  provide  infonnation  about  the  design  of  simulation 
training,  perceived  realism,  and  apparent  effectiveness.  Such  measures 
can  be  used  to  subjectively  assess  constructs  associated  with  perfonnance, 
such  as  stress,  cognitive  workload,  and  fatigue.  However,  the  use  of  only 
subjective  performance  measures  is  inadequate  for  evaluating  the  actual 
effectiveness  of  training  (Bell  &  Waag,  1998).  Furthennore,  reliance  on 
subjective  measures  exclusively  is  perilous  for  safety  critical  tasks  because 
subjective  and  objective  measures,  although  related,  often  diverge  (Andre 
&  Wickens,  1995). 

3 .  Type  of  training:  Simulation  training  can  cover  part  of  the  task  or  the  whole 
task  (see  scope  of  training).  However,  a  combination  of  the  task  scope, 
CTA,  training  goals,  and  performance  measures  is  necessary  to  ascertain 


210 


Reviews  of  Human  Factors  and  Ergonomics,  Volume  8 


which  specific  parts  of  the  task  should  be  simulated  or  if  training  for  the 
entire  task  is  appropriate.  In  addition,  the  same  information  is  needed  to 
decide  if  it  is  appropriate  to  include  incidents  or  mimic  EPCs  to  help  clini¬ 
cians  learn  to  prioritize  important  tasks. 

a.  Individual  and  team  training:  By  reviewing  the  CTA,  goals,  perfor¬ 
mance  measures,  best  practices,  and  local  practices,  one  can  ascertain  if 
task  steps  require  more  than  one  person  to  be  successfully  completed. 
In  health  care,  simulation  training  for  teams  has  not  been  widely 
addressed  (Rosen,  2008),  although  there  are  notable  exceptions  (e.g., 
Baker  et  al„  2005;  Beaubien,  2004;  Hamman,  2004;  Howard,  Gaba,  Fish, 
Yang,  &  Sarnquist,  1992;  Rosen  et  ah,  2008;  Small  et  ah,  1999;  Wayne 
et  ah,  2008).  For  an  overview  on  simulation  team  training  in  health 
care,  see  Baker  et  al.  (2005).  An  overview  about  medical  team  training  is 
available  in  this  volume  (Xiao,  Parker,  &  Manser,  in  press). 

b.  PTT  and  whole-task  training:  PTT  covers  critical  steps  or  sub-steps  of 
the  activity,  whereas  whole-task  training  covers  the  entire  activity  (e.g., 
Wickens,  Gordon,  &  Fiu,  1998).  A  simulator  for  PTT  will  have  lower 
complexity  and  likely  lower  overall  physical  realism,  given  the  focus  on 
specific  critical  steps,  than  a  simulator  used  for  whole-task  training.  An 
example  of  PTT  is  simulation  of  the  technical  and  procedural  skills  for 
suturing  while  excluding  the  diagnostic  assessment  and  clinician  inter¬ 
action  with  the  patient.  PTT  is  suited  for  independent  components  of 
an  activity  that  require  repeated  practice  to  learn.  Without  the  CTA 
determination  of  what  constitutes  an  independent  grouping  of  task 
steps,  PTT  development  is  difficult.  For  PTT,  part-task  simulators  (Per¬ 
kins,  2007)  or  scaled-world  simulators  (Gray,  2002)  are  ideal  (the  term 
part-task  simulation  originates  in  the  medical  literature,  and  the  term 
scaled  world  simidator  is  commonly  used  in  the  psychological  training 
literature).  In  both  cases,  only  “a  subset  of  the  functional  relationships 
found  in  a  complex  task  environment”  (Gray,  2002,  p.  208)  is  selected  to 
train  specific  skills.  Consequentially,  part-task  simulators  are  typically 
relatively  simple  anatomical  models  for  training  specific  technical  and 
procedure  skills  (Perkins,  2007;  Rosen,  2008).  Multiple  simulation  tools 
can  be  combined  to  cover  distinct  tasks  (Bradley,  2006);  for  example, 
training  for  taking  a  medical  history  could  use  a  standardized  patient  as 
the  simulation  tool,  whereas  technical  and  procedural  skills  for  sutur¬ 
ing  could  be  performed  on  a  pig’s  foot. 

c.  Variable  priority  training  (VPT):  VPT  uses  a  whole  task  training 
approach  with  the  goal  of  prioritizing  components  of  a  task  by  control¬ 
ling  the  allocation  of  attention  (Wickens  et  al.,  1998).  The  CTA,  train- 
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ing  goals,  and  other  information  are  needed  to  determine  if  training 
should  incorporate  VPT.  Training  with  PTT  and  VPT  in  anesthesiology 
leads  to  improved  performance  in  the  simulator  (Johnson  et  ah,  2008). 

4.  Simulation  tool/ technology:  The  selection  of  an  appropriate  simulation 
tool  requires  the  definition  of  the  scope  of  the  clinical  task,  its  psychologi¬ 
cal  operations,  and  the  training  goals  and  performance  measures.  Available 
simulation  technologies  were  described  in  Types  of  Simulators  section. 

5.  Feedback  method:  Immediate  performance  feedback  after  task  completion 
provides  opportunities  for  learning  (e.g.,  Grenvik  et  al.,  2004),  whereas 
structured  feedback  can  be  provided  through  after-action  reviews  (Fan¬ 
ning  &  Gaba,  2007).  Feedback  methods  include  simulator  based,  auto¬ 
matic  assessment  of  performance  in  real  time  or  by  delayed  performance 
summary;  instructor  assessment  in  real  time  providing  feedback  during 
simulation  interruptions  or  simultaneously;  and  video-based  debriefing 
performed  at  the  conclusion  of  the  simulation  (Dismukes  &  Smith,  2001; 
Rudolph,  Simon,  Raemer,  &  Eppich,  2008).  Feedback  during  simulation 
provides  one  of  the  best  opportunities  for  learning  since  in  clinical  prac¬ 
tice,  immediate  feedback  is  rare  (Durso  &  Drews,  2010). 

Other  factors  relevant  to  the  effectiveness  of  training  are  not  explicitly  covered  in  this 
framework.  Among  these  are  trainee  characteristics,  such  as  background  knowledge, 
ability,  and  motivation  (Baldwin  &  Ford,  1988).  Participant  motivation  can  be  bolstered 
by  providing  a  clear  purpose  and  set  of  expectations  for  training  (Baldwin  &  Ford,  1988) 
and  by  presenting  training  as  an  opportunity  instead  of  an  evaluation  (Salas  et  al.,  2012). 
Incorporating  progressively  increasing  difficulty  levels  and  repeated  practice  are  recom¬ 
mended  for  effective  training  (Issenberg  &  Scalese,  2008).  Finally,  organizational  factors 
may  impede  or  impair  the  success  of  simulation  training.  Salas  et  al.  (2012)  put  forward 
a  comprehensive  approach  prior  to  developing  training  called  a  training  needs  analysis 
to  determine  “expected  learning  outcomes,  guidance  for  training  design  and  delivery, 
ideas  for  training  evaluation,  and  information  about  the  organizational  factors  that  will 
likely  facilitate  or  hinder  training  effectiveness”  (p.  80). 

Overall,  the  aforementioned  framework  is  conceptual  and  provides  guidance  to  help 
identify  the  psychological  operations  for  the  clinical  task  that  guide  the  selection  of  the 
simulation  technology.  However,  its  application  does  not  always  guarantee  positive 
transfer  since  the  amount  of  psychological  correspondence  necessary  for  positive  trans¬ 
fer  is  not  known  (other  than  the  generic  statement  of  as  much  as  possible)  and  must  be 
inferred  (see  Inferring  Transfer  of  Training). 

Training  myths.  In  addition  to  factors  that  increase  the  likelihood  of  training  effec¬ 
tiveness,  there  are  intuitively  appealing  but  misguided  factors  that  may  decrease  training 
effectiveness.  Two  myths,  cognitive  learning  styles  and  high-(physical)-fidelity  simula¬ 
tion,  are  dispelled. 
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The  pervasive  belief  that  instruction  is  most  effective  when  it  conforms  to  an  indi¬ 
vidual’s  cognitive  or  learning  style,  such  as  kinesthetic,  auditory,  or  visual,  is  a  myth 
(Pashler  &  McDaniel,  2008;  Riener  &  Willingham,  2010).  The  research  and  applica¬ 
tions  of  learning  styles  often  focus  on  teaching  children  (e.g.,  Riener  &  Willingham, 
2010)  but  have  also  been  suggested  for  tailoring  simulation  training  to  the  preferred 
learning  styles  of  clinicians  (e.g.,  Newbie  &  Entwistle,  2009;  Weaver  et  ah,  2010). 
Unfortunately,  there  is  almost  no  empirical  evidence  that  cognitive/learning  styles 
should  be  taken  in  account  because,  regardless  of  individual  preferences,  information 
presented  in  multiple  modalities  consistently  leads  to  the  best  acquisition  of  knowl¬ 
edge  (Pashler  &  McDaniel,  2008;  Riener  &  Willingham,  2010).  Given  the  absence  of 
empirical  support  for  learning  styles  in  the  education  and  psychology  literature,  it  is 
unlikely  that  tailoring  the  modalities  of  instruction,  such  as  the  simulation  tool,  to 
preferred  learning  styles  will  increase  training  effectiveness.  Instead,  we  recommend 
tailoring  simulation  to  the  training  goals,  specifically,  psychological  fidelity,  which  is 
empirically  supported. 

The  second  myth  is  the  belief  that  high-fidelity  simulation  (i.e.,  strong  physical  real¬ 
ism  of  the  simulation  tool  and  environment)  is  by  itself  sufficient  to  make  training 
effective.  If  the  physical  realism  in  the  simulation  does  not  correspond  to  the  psycho¬ 
logical  aspects  of  performing  the  actual  task,  high  physical  fidelity  even  may  be  detri¬ 
mental  to  training  (Kozlowski  &  DeShon,  2004).  The  erroneous  emphasis  on 
high-fidelity  simulation  haunted  aviation  for  decades,  increasing  the  cost  of  simula¬ 
tion,  limiting  its  availability,  and  perhaps  even  diminishing  its  effectiveness  (see  Alessi, 
2000;  Hopkins,  1974). 

Transfer  of  Training 

The  most  important  objective  of  training  is  successful  transfer;  that  is,  the  knowl¬ 
edge,  skills,  and  abilities  acquired  during  training  translate  into  improved  real-world 
performance  (Baldwin  &  Ford,  1988;  Salas  et  al.,  1998,  2012).  Transfer  of  training  is 
suggested  by  improved  performance  in  the  simulator,  but  that  improvement  is  not 
the  definitive  indicator.  To  validate  simulation  performance  evaluations,  a  compari¬ 
son  with  real-world  performance  is  required  (Devitt,  Kurrek,  Cohen,  &  Cleave-Hogg, 
2001;  Dong  et  al.,  2010;  Johnson,  Guediri,  Kilkenny,  &  Clough,  2011;  Schout  et  al., 
2010).  Training  that  leads  to  increased  real-world  performance  is  a  result  of  positive 
transfer  (Osgood,  1949),  and  high  psychological  fidelity  is  central  for  positive  transfer 
to  occur  (Adams,  1979;  Kozlowski  &  DeShon,  2004).  When  training  neither  increases 
nor  decreases  real-world  performance,  there  is  zero  transfer.  Negative  transfer 
(Osgood,  1949)  involves  training  that  decreases  real-world  performance  and  results 
from  an  incompatibility  in  applying  the  knowledge,  skills,  and  abilities  acquired  dur¬ 
ing  training  to  the  real  world  (Lui,  Blickensderfer,  Macchiarella,  &  Vincenzi,  2008). 
Negative  transfer  can  also  result  from  mismatched  or  incorrect  psychological  fidelity, 
for  example,  learning  the  incorrect  sequence  of  steps,  irrelevant  tasks,  distracting 
high  physical  fidelity,  or  equipment  differences,  which  results  in  habit  interference  (Lui 
et  al.,  2008). 
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The  concept  of  transfer  of  training  represents  a  continuum  (Lui  et  al,  2008).  Positive 
transfer  from  simulation  training  to  the  real  world  is  often  partial,  especially  during  the 
early  stages  of  training.  Some  but  not  all  skills  and  knowledge  successfully  transfer  from 
the  simulation  to  the  real  world,  and  the  rates  of  learning  may  be  lower  than  real-world 
training  (Schmidt-Panos  8r  Scerbo,  2008).  For  many  tasks  in  health  care,  one  of  the  pri¬ 
mary  goals  of  simulation  training  is  not  just  the  initial  acquisition  of  knowledge  and 
skills  but  also  their  retention.  Although  simulation  has  been  proposed  to  improve  skill 
retention  (Ziv  et  al.,  2006),  longitudinal  assessment  of  performance  in  health  care  is 
uncommon  (although  see  Wayne  et  al.,  2006). 

Transfer  of  simulation  training  to  clinical  practice.  Literature  reviews  and  meta¬ 
analyses  indicate  that  simulation  training  in  health  care  is  highly  effective  (McGaghie  et 
al.,  2010, 2011;  Ross  et  al.,  2012).  However,  the  majority  of  reviewed  studies  used  perfor¬ 
mance  measures  limited  to  the  simulator  and  failed  to  assess  actual  clinical  practice.  Still, 
positive  transfer  of  simulation  training  has  been  found  in  a  variety  of  activities,  shown 
in  Table  5.2. 

The  limited  empirical  assessment  of  transfer  of  training  in  the  health  care  setting 
reflects  various  challenges  associated  with  measuring  real-world  clinical  perfor¬ 
mance.  Measuring  human  performance  can  be  expensive  and  time  consuming, 
require  institutional  support,  and  have  methodological  confounds  (e.g.,  uncontrollable 
factors,  such  as  patient  health  and  the  unpredictability  of  responses  to  treatment;  Gaba, 
2004;  Schout  et  al.,  2010).  Another  challenge  arises  from  ethical  and  legal  considerations 
of  continued  use  of  traditional  training  methods  with  more  effective  methods  being 
available  (Ziv  et  al.,  2006). 

Theories  of  transfer  of  training.  Theories  of  transfer  of  training  can  be  contrasted 
from  two  positions  (Lathan,  Tracey,  Sebrechts,  Clawson,  &  Higgins,  2002;  Singley  & 
Anderson,  1989): 

1 .  General  transfer:  Transfer  is  broad  and  generalizes  from  one  task  to  another, 
even  when  unrelated  (“Doctrine  of  Formal  Discipline”;  Angell,  1908,  as 
cited  in  Lathan  et  al.,  2002).  For  example,  the  knowledge,  skills,  and  abili¬ 
ties  acquired  from  simulation-based  training  for  suturing  a  cut  on  a  pig’s 
foot  generalizes  to  suturing  cuts,  in  a  variety  of  locations,  on  patients. 

2.  Specific  transfer:  The  two  tasks  must  be  identical  for  transfer  to  occur 
(“Theory  of  Identical  Elements”;  Thorndike,  1906,  as  cited  in  Lathan  et  al., 
2002).  An  example  of  specific  transfer  is  that  simulation  training  for  sutur¬ 
ing  using  a  pig’s  foot  transfers  only  to  the  exact  same  task  and  simulation 
tool,  that  is,  suturing  using  a  pig’s  foot. 


Both  theories  have  flaws:  Evidence  for  general  transfer  is  scarce  because  skill  general¬ 
ization  between  tasks  with  unrelated  psychological  operations  is  difficult  to  establish 
(Singley  &  Anderson,  1989).  Specific  transfer  implies  that  it  is  necessary  to  train  under 
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Table  5.2.  Examples  of  Positive  Transfer  of  Simulation  Training 


Tasks 


Anesthesiology:  Intubation  and  ventilation 
techniques,  correct  application  of  pressure 
to  prevent  regurgitation  during  anesthesia, 
and  management  of  incidents  and  adverse 
events. 

Bronchoscopy:  Diagnostic  viewing  of  the 
airways  and  the  lungs. 

Cardiac  arrests  (resuscitation  when  the  heart 
stops  beating). 

Central  line  placement:  A  catheter  inserted 
near  the  heart,  used  to  infuse  medications 
and  draw  blood. 


Diagnosis  of  heart  murmurs. 

Dystocia:  A  complication  in  birth  in  which  the 
baby’s  shoulder  blocks  delivery. 

Surgical  procedures:  Colonoscopy;  robot- 
assisted  techniques,  such  as  laparoscopic 
surgery;  etc. 
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every  possible  condition  that  could  occur  in  the  real  world,  which  is  impossible  (Singley 
&  Anderson,  1989).  Furthermore,  specific  transfer  requires  high  physical  fidelity,  a 
requirement  that  is  at  odds  with  research  demonstrating  the  effectiveness  and  impor¬ 
tance  of  psychological  fidelity. 

The  theory  of  transfer  appropriate  processing  (Morris,  Bransford,  &  Franks,  1977) 
falls  between  the  general  transfer  and  specific  transfer  extremes.  It  predicts  that  when  the 
initial  task  (i.e.,  simulation  training)  supports  the  psychological  operations  of  the  sec¬ 
ond  task  (i.e.,  clinical  practice  in  the  real  world),  then  positive  transfer  occurs.  In  order 
to  develop  training  that  is  likely  to  transfer  to  a  reasonable  range  of  real-world  condi¬ 
tions,  skills  can  be  practiced  repeatedly  with  performance  feedback  until  they  approach 
automaticity;  separate  training  can  be  conducted  for  independent,  critical  steps  using 
PTT;  and  a  final  training  can  be  conducted  under  conditions  that  mimic  the  operational 
environment  (i.e.,  simulated  incidents  and  adverse  events  and  EPCs).  Compared  to  sim¬ 
ulation  training  with  transfer  appropriate  processing,  traditional  training  does  not  typi¬ 
cally  permit  repeated  practice  under  safe  and  standardized  conditions. 

Psychological  fidelity  and  transfer.  An  important  lesson  in  developing  effective  sim¬ 
ulation  training  in  health  care  can  be  taken  from  aviation.  For  decades,  aviation  training 
singularly  focused  on  high-fidelity  simulation  and  failed  to  produce  tangible  benefits 
(Alessi,  2000).  Questions  that  were  raised  about  the  significant  expenses  and  lack  of 
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benefit  from  aviation  simulators  with  “bells  and  whistles”  (Hopkins,  1974)  went 
unheeded.  Empirical  work  on  the  transfer  of  training  in  aviation  repeatedly  demon¬ 
strated  the  importance  of  psychological  fidelity  across  numerous  tasks  (Adams,  1979; 
Hays,  Jacobs,  Prince,  &  Salas,  1992;  Kozlowski  &  DeShon,  2004).  Only  when  high  physi¬ 
cal  fidelity  also  has  psychological  relevance  to  the  task  has  it  been  shown  to  improve 
transfer  of  training,  for  example,  haptic  and  visual  feedback  for  laparoscopic  surgery 
(Hays  et  al.,  1992).  For  simulation  training  to  become  widespread  in  health  care,  it  needs 
to  be  cost  effective.  For  that  reason,  the  same  pursuit  of  naive  realism  as  in  aviation  needs 
to  be  avoided.  Fortunately,  more  and  more  researchers  in  health  care  identify  the  impor¬ 
tance  of  psychological  fidelity  as  a  critical  dimension  for  effective  training  (Beaubien, 
2004;  Maran  &  Glavin,  2003;  Scerbo  &  Dawson,  2007).  Still,  the  allure  of  naive  realism  in 
simulation  is  enduring  (Hook,  2004;  Smallman  &  St.  John,  2005). 

Quality  of  Care  and  Clinical  Performance 

The  overarching  goal  of  simulation  training  is  the  safe  and  effective  delivery  of  high 
quality  care.  In  contrast  to  clinical  performance,  quality  of  care  is  a  long-term,  multi¬ 
dimensional  metric  that  requires  multiple  measures,  including  mortality,  clinical  out¬ 
comes,  patient  functional  status,  and  patient  quality  of  life  (see  Clancy,  1998).  Because 
a  multitude  of  factors  impact  quality  of  care  and  clinical  outcomes,  it  is  difficult  to 
determine  if  simulation  training  improves  performance  on  all  of  these  measures. 

The  goal  of  simulation  training  is  improved  quality  of  care  and  clinical  outcomes, 
but  each  is  difficult  to  assess.  Hence,  we  contend  that  a  more  easily  measured,  imme¬ 
diate,  and  directly  task-relevant  level  of  analysis  is  assessment  of  clinical  perfor¬ 
mance.  Clinical  performance  is  a  predictor  of  risk  for  suboptimal  clinical  outcomes 
and  thus  quality  of  care,  just  as  “near  misses”  and  “close  calls”  in  aviation  are  risk 
predictors  for  accidents  and  fatalities.  In  health  care,  “near  misses”  can  be  inferred 
using  clinical  performance  measures,  such  as  nonadherence  to  best  practices  and 
error  (Drews,  in  press). 

To  conceptualize  and  understand  the  relationships  between  quality  of  care,  clinical 
outcomes,  and  clinical  performance,  we  use  Simon’s  (1996)  theory  of  complex,  hierar¬ 
chical  systems.  In  a  hierarchical  system,  there  are  interrelated  systems  with  each  level 
influenced  by  all  of  the  prior  levels  (Simon,  1996).  As  the  level  of  analysis  increases,  the 
number  of  related  systems  and  subsystems  grows,  increasing  the  complexity.  Here,  the 
highest  level  of  analysis  is  quality  of  care  (incorporating  clinical  outcomes  and  clinical 
performance),  followed  by  the  midlevel  of  clinical  outcomes  (including  clinical  perfor¬ 
mance),  and  the  lowest  level  of  clinical  performance  (Table  5.3). 

In  health  care,  the  time  between  the  levels  of  analysis  varies  by  orders  of  magnitude. 
The  majority  of  clinicians  do  not  receive  immediate  feedback  on  their  clinical  perfor¬ 
mance,  which  impacts  clinical  outcomes  and  influences  the  quality  of  care.  The  epochs 
and  open-loop  feedback  in  health  care  can  be  contrasted  with  aviation,  whereby  pilots 
generally  receive  immediate  or  close-to-immediate  feedback  on  the  majority  of  their 
actions,  that  is,  closed-loop  feedback. 

Clinical  outcomes  and  quality  of  care  are  influenced  by  multiple  tasks  or  procedures 
and  other  factors,  such  as  patient  health.  In  addition,  there  is  a  limited  subset  of  tasks 
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Table  5.3.  Clinical  Performance:  Hierarchical  Levels  of  Analysis 


Level  of  Analysis 

Dependent  Variables 

Time  Scale 

Quality  of  care 

A  multidimensional  measure 
that  takes  into  account  clinical 
outcomes,  mortality  rates,  and 
patient  reported  data,  such  as 
functional  status  and  quality 
of  life. 

Typically  weeks,  months,  and 
years.  In  emergency  care 
and  other  specialized  areas, 
such  as  critical  care  or 
anesthesiology,  there  may  be  a 
more  direct  relationship  to  lower 
levels,  with  clinical  performance 
affecting  clinical  outcomes 
immediately. 

Clinical  outcomes  and 
adverse  events 

Medical  and  laboratory  record 
abstraction  of  physiological 
and  diagnostic  tests,  for 
example,  infections.  Clinical 
outcomes  and  adverse  events 
can  be  derived  from  objective 
classification  and  subjective 
classification  by  expert  and 
nonexpert  clinicians. 

Days,  weeks,  and  longer. 

Clinical  performance 

Observations  or  video  recording 
of  adherence  to  best  practices, 
local  practice  protocols, 
incidents,  and  error.  May  also 
be  inferred  from  item  and 
equipment  use. 

Seconds,  minutes,  and  hours. 

Note:  Hierarchical  levels  of  analysis  for  clinical  performance,  dependent  measures,  and  time  scale. 
The  highest  level  of  analysis  is  quality  of  care,  followed  by  clinical  outcomes  and,  last,  clinical  per¬ 
formance.  Over  time,  each  level  integrates  all  elements  of  the  lower  levels.  That  is,  quality  of  care 
is  associated  with  clinical  outcomes  and  clinical  performance  because  adherence  to  best  practices 
increases  the  likelihood  of  positive  clinical  outcomes,  which  are  indicators  and  causal  factors  for 
high  quality  of  care. 


and  activities  in  health  care  with  immediate  consequences.  Exceptions  include  specific 
situations  in  the  areas  of  emergency  care,  critical  care,  respiratory  therapy,  surgery,  and 
anesthesiology.  In  these  areas,  clinical  outcomes  and  mortality  may  be  on  the  time  scale 
of  seconds,  minutes,  or  hours. 

To  limit  the  scope  of  the  hierarchical  system,  other  potentially  relevant  factors  (e.g., 
the  operational  environment  and  social/cultural  factors)  are  excluded.  Clinical  perfor¬ 
mance  could  be  further  reduced  to  the  impact  of  adherence  or  nonadherence  to  best 
practices.  In  health  care,  the  relationship  between  time  and  levels  of  analysis  is  more 
formally  described  using  the  two  propositions  for  nearly  decomposable  systems  (Simon, 
1996): 

1.  Over  a  short  time  scale,  levels  are  mostly  independent  of  each  other. 
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2.  Over  a  long  time  frame,  the  outcome  at  the  higher  levels  depends  upon  the 
aggregation  of  the  outcomes  of  the  levels  below. 

Clinical  Performance 

The  hierarchical  level  of  analysis  is  a  theoretical  framework  that  explains  how  clinical 
performance  acts  as  a  precursor  to  clinical  outcomes  and  how  clinical  outcomes  impact 
quality  of  care.  Clinical  performance  is  the  common  level  of  analysis  for  both  simulation 
training  and  the  real  world.  Methodological  considerations  for  simulation  training 
include  skill  retention,  type  of  training  (e.g.,  repeated  practice  and  incidents,  VPT,  and 
PTT),  and  clinical  performance  measures. 

For  simulation  training  to  be  effective,  it  must  both  transfer  to  the  real  world,  and 
the  knowledge,  skills,  and  abilities  acquired  in  training  must  be  retained  over  time. 
Practice  is  fundamental  to  achieving  expert  performance  (Ericsson,  Krampe,  &  Tesch- 
romer,  1993).  Achieving  expert  performance  may  require  10,000  hr  of  practice  (Simon, 
1996). 

Several  theoretical  frameworks  on  skill  acquisition  (e.g.,  Fitts,  1964;  Rasmussen, 
1986)  focus  on  the  importance  of  practice,  not  necessarily  transfer.  Nonetheless,  these 
frameworks  provide  a  solid  foundation  for  skill  acquisition  and  retention  that  is  missing 
from  the  transfer  theories  discussed  earlier.  All  of  these  frameworks  propose  a  three- 
stage  learning  process  with  stages  consisting  of  declarative  knowledge  (learn¬ 
ing  stage),  procedural  knowledge  (consolidation  stage),  and  a  combination  of  declara¬ 
tive  and  procedural  knowledge  (tuning  stage)  (Kim,  Ritter,  &  Koubek,  2013).  The  third 
stage  of  learning  constitutes  expert  performance.  Kim  et  al.  (2013)  propose  that  each 
stage  of  learning  has  different  properties  for  forgetting.  Because  knowledge  in  the  first 
stage  of  learning  is  strictly  declarative,  there  is  potential  for  memory  failures,  with  much 
of  the  learning  from  practice  being  lost.  With  sufficient  practice,  knowledge  becomes 
increasingly  procedural  and  thus  more  resilient  to  forgetting. 

Another  issue  in  training  is  the  spacing  between  learning,  that  is,  should  it  be  massed 
(repeated  back-to-back  training)  or  distributed  practice  (more  time  between  training 
intervals)  (Cepeda,  Paschler,  Vul,  Wixted,  &  Rohrer,  2006)?  The  type  of  task,  goals,  and 
performance  measures  should  be  used  to  determine  whether  practice  is  massed  or  dis¬ 
tributed.  Massed  practice  may  lead  to  better  acquisition  and  retention  for  skills  that  are 
primarily  procedural  (Cepeda  et  al.,  2006),  such  as  suturing.  For  retention  of  declarative 
knowledge,  distributed  practice  is  generally  superior  to  massed  practice.  However,  the 
length  of  the  spacing  intervals  is  uncertain  (Cepeda  et  al.,  2006).  Because  simulation 
training  may  consist  of  a  task  with  procedural  and  declarative  components,  it  may  be 
advisable  to  separate  the  task  components  using  PTT.  Massed  practice  on  procedural 
and  declarative  components  can  be  trained  on  a  distributed  schedule. 

Based  on  the  theories  of  learning,  skill  acquisition,  and  forgetting,  simulation  train¬ 
ing  should  reach  at  least  the  second  stage  of  learning  and  realize  a  minimum  level  of 
proficiency  and  reduced  skill  decay  to  prevent  memory  failures.  To  determine  if  the 
second  stage  of  learning  is  reached,  simulation  practice  requires  repetition  until  a 
minimum  performance  criterion  is  reached.  However,  it  is  unlikely  that  simulation 
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training  will  ever  be  extensive  enough  to  achieve  expert  performance,  the  third  stage 
of  learning. 

Measurement  Methods 

Multiple  methods  and  measures  for  evaluating  clinical  performance  exist,  such  as  video 
recording,  observation,  metrics  provided  by  the  simulation  tool  (e.g.,  chest  compression 
force  for  administering  cardiopulmonary  resuscitation  on  a  mannequin),  and  inventory 
records  (for  item  use).  Next,  the  advantages  and  disadvantages  of  different  methods  of 
measuring  clinical  performance  are  discussed. 

An  advantage  of  video  recording  is  that  performance  can  be  coded  by  multiple 
raters;  thus  psychometric  properties  of  measures,  such  as  inter-rater  reliability,  can 
be  quantified  and  reviews  of  videos  can  be  used  to  provide  trainees  with  explicit 
performance  feedback.  Another  advantage  is  that  coding  can  be  performed  using 
computer  vision  algorithms,  which  may  reduce  the  need  for  human  raters,  may  lower 
costs,  and  may  even  provide  automated,  structured  performance  feedback  (e.g., 
Zhang  &  Li,  2010).  Although  video  recording  is  easily  employed  in  simulation  training, 
in  a  clinical  setting,  concerns  of  patient  privacy  and  potential  legal  liability  complicate  its 
use.  Observation  is  typically  the  most  common  method  of  measurement. 

With  observation,  performance  must  be  coded  in  real  time;  thus  if  clinician  actions 
are  missed  by  the  coder,  they  cannot  be  reviewed.  Because  there  is  no  video  recording, 
observation  cannot  provide  the  same  level  of  detail  in  the  performance  feedback. 
Observation  and  video  coding  have  the  same  drawback:  requiring  a  rater,  or  ideally  mul¬ 
tiple  raters,  with  sufficient  clinical  knowledge  and  expertise  to  conduct  valid  and  reliable 
coding. 

In  actual  clinical  practice,  item  use  provides  a  simple,  albeit  approximate,  measure  of 
clinical  performance.  Item  use  can  be  informative  for  tasks  that  occur  with  regularity 
(e.g.,  oral  care  and  central  line  maintenance).  When  the  use  of  items  is  indicative  of 
adherence  to  best  practices,  lack  of  item  consumption  allows  potential  identification  of 
nonadherence.  However,  knowing  a  task  was  performed  is  not  sufficient  to  determine  if 
best  practice  standards  were  followed.  In  addition,  item  consumption  relies  on  the  accu¬ 
racy  and  capabilities  of  the  inventory  system.  If  the  health  care  facility  has  an  inventory 
system  linking  items  to  specific  patients,  then  inferences  at  the  patient  level  are 
possible. 

Operationalizing  clinical  performance.  Clinical  performance  measures  can  be  oper¬ 
ationalized  using  adherence  to  best  practices  and  local  practice  protocols,  the  frequency 
and  type  of  error,  item  use,  and  self-report  measures.  Different  clinical  performance 
measures  may  overlap.  For  example,  some  best  practices  may  be  included  in  local  prac¬ 
tice  protocols,  and  deviations  from  evidence-based  practices  can  be  considered  as  error. 

Specific  metrics  of  clinical  performance  should  be  selected  using  evidence  based 
practices,  local  practice  protocols,  training  goals,  and  the  results  of  the  CTA.  To  have 
a  well-defined  relationship  between  clinical  performance  and  outcomes,  best  practices 
that  are  empirically  supported  should  be  used.  There  are  wide  variations  in  the  strength 
of  the  evidence  for  best  practices.  These  levels  range  from  well-established  empirical 
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evidence,  followed  by  expert  opinion  and  manufacturer  guidelines  or  recommendations 
(Makic  et  al.,  2011).  If  “incidents”  are  part  of  simulation  training,  there  may  be  specific 
best  practices  for  particular  situations,  which  may  also  depend  on  the  local  practices, 
training  goals,  and  the  CTA.  Incidents  maybe  rare  in  clinical  performance  (although  see 
Medmarx  for  a  medication  error  database;  http://www.medmarx.com),  so  measurement 
of  transfer  performance  for  incidents  may  not  be  possible. 

Error  may  consist  of  best  practice  nonadherence  but  may  also  reflect  omission  of 
procedure  steps,  forgetting  items,  and  other  actions  that  are  implied  in  best  practices. 
There  are  causal  descriptions  of  error,  which  are  more  detailed  than  adherence  or  non¬ 
adherence  to  a  standard.  Error  can  be  classified  using  the  Reason  (1990)  taxonomy  (see 
also  Drews,  2011;  Drews,  Wallace,  Benuzillo,  Markewitz,  &  Samore,  2012;  Morrow  et  al., 
2005): 

1.  Slip:  Breakdowns  in  selecting  or  recognizing  appropriate  actions 

2.  Lapse:  Memory  or  attention  failures 

3.  Mistake:  Incorrect  choices  or  wrong  objectives 

4.  Violation:  Conscious  avoidance  of  normal  practices,  that  is,  intentional 
“shortcuts.” 

Inferring  transfer  of  training.  From  a  methodological  perspective,  an  ideal  research 
design  directly  compares  real-world  performance  from  simulation  training  to  that  from 
traditional  training  (see  Strayer  &  Drews,  2003).  However,  this  type  of  research  design 
may  not  be  possible  due  to  considerations  outlined  earlier.  Fortunately,  there  are  several 
alternatives  to  infer  transfer  of  training. 

Simulation  validation.  Clinical  performance  measures  for  the  simulation  can  be 
validated  by  comparing  clinical  performance  in  the  simulation  and  clinical  performance 
in  the  real  world  (e.g.,  Schout  et  al.,  2010).  There  are  numerous  possible  confounds,  for 
example,  inconsistencies  between  real  patients  and  the  simulation  tool.  Nevertheless, 
comparable  performance  in  simulation  and  clinical  practice  implies  that  simulation- 
based  performance  assessment  is  a  valid  proxy  for  real  clinical  practice.  Furthermore, 
validation  makes  it  possible  to  control  for  traditional  training  by  evaluating  perfor¬ 
mance  in  the  simulator. 

A  complementary  approach  toward  validation  is  to  compare  simulator  performance 
of  participants  with  different  levels  of  expertise.  Demonstration  of  group  differentiation 
based  on  skill  levels  can  serve  as  a  validation  of  the  simulated  scenarios  (Murray  et  al., 
2007). 

Quasi-transfer.  Related  to  validation  of  simulation  performance,  quasi-transfer  is  a 
method  for  testing  the  generalizability  of  performance  by  using  one  simulation  method 
for  training  and  a  second  one  for  transfer.  An  example  of  quasi-transfer  is  using  virtual 
reality  for  surgery  training  with  surgery  on  human  cadavers  to  evaluate  transfer  (e.g., 
Schout  et  al.,  2010).  In  aviation,  quasi-transfer  has  been  demonstrated  to  be  predictive 
of  real-world  performance  (Taylor  et  al.,  1993). 
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Training  effectiveness.  There  is  also  a  set  of  descriptive  methods  for  comparing 
training  effectiveness  in  terms  of  time  or  cost  using  ratios,  that  is,  percentage  transfer, 
transfer  effectiveness,  and  incremental  training  effectiveness  (Roscoe  &  Williges,  1980). 
Developed  for  aviation,  these  methods  allow  quantifying  the  time  and  costs  of  simula¬ 
tion  versus  real  world  in  terms  of  training  effectiveness.  However,  in  health  care,  ethical 
considerations  and  the  quantification  of  the  expenses  of  clinician  “learning”  on  patients 
is  a  challenge  to  this  approach. 

Clinical  outcomes.  Clinical  outcomes  can  be  measured  via  abstraction  of  medical 
records,  including  laboratory,  physiological,  and  diagnostic  tests.  Some  clinical  out¬ 
comes  are  rare,  so  years  of  patient  data  may  be  required  to  determine  the  effectiveness  of 
simulation  training  for  a  single  institution.  Past  data  on  clinical  outcomes  can  be  used  to 
compare  with  simulation  training  data,  but  such  simple  pre-  and  post-training  research 
designs  have  potential  for  numerous  confounds,  such  as  improved  equipment,  increased 
awareness  of  best  practices  due  to  continuing  education,  and  changes  in  local  practices, 
protocols,  and  staffing,  to  name  a  few. 

Objective,  systematic  methods  should  be  used  to  select  clinical  outcome  measures 
(see  Campbell,  Braspenning,  Hutchinson,  &  Marshall,  2002).  Any  subjective  classifica¬ 
tion  of  clinical  outcomes  should  be  interpreted  with  caution  because  the  frequency  and 
severity  of  negative  clinical  outcomes  may  be  underestimated.  For  example,  Lin  et  al. 
(2010)  found  that  subjective  classification  of  CLABSI  rates  in  hospitals  were  as  much  as 
four  times  lower  than  objective  classification.  Also,  abstracted  information  from  medical 
records  reviewed  by  expert  raters  was  inconsistent  and  tended  to  systematically  under¬ 
report  the  severity  of  diseases  and  illness  (Luck,  Peabody,  &  Dresselhaus,  2000). 

Similar  to  clinical  performance,  objective  methods  for  determining  clinical  outcomes 
is  essential  to  the  validity  and  reliability  of  the  measurement  construct.  Comparisons  in 
the  rates  of  negative  clinical  outcomes  between  health  care  facilities  should  be  inter¬ 
preted  carefully  (Luck  et  al.,  2000).  Hospitals  that  frequently  perform  high-risk  proce¬ 
dures  are  more  likely  to  have  sicker  patients  and  higher  rates  of  negative  clinical  outcomes 
compared  to  hospitals  that  perform  fewer  or  none  of  the  same  high-risk  procedures. 
Despite  the  limitations  of  measuring  clinical  outcomes,  they  are  an  essential  component 
in  a  cost  analysis  of  simulation  training. 


COST  ANALYSIS  OF  SIMULATION  TRAINING 

Compelling  data  exist  indicating  that  simulation  training  can  save  money  compared 
with  traditional  training.  One  possible  reason  for  the  low  adoption  rate  of  simulation 
training  is  its  perceived  high  cost.  In  addition  to  the  monetary  expense  of  the  simula¬ 
tion  tool,  other  expenses  include  the  costs  of  training  development,  staff  and  instructor 
time,  and  evaluation.  Few  studies  report  the  costs  of  simulation  training  (Zendejas, 
Wang,  Brydges,  Hamstra,  &  Cook,  2013).  Simulation  systems  can  range  in  cost  from  less 
than  $5,000  for  laparoscopic  simulators  to  over  $300,000  for  sophisticated  anesthesia 
simulators  (Issenberg  et  al.,  1999).  Student  trainees  increase  patient  safety  risks. 
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Although  traditional  training  protocols  involve  the  guidance  of  an  experienced  mentor 
(Haque  &  Srinivasan,  2006),  the  trainee  may  not  always  follow  the  protocol  (Haluck  & 
Krummel,  2000). 

In  high  risk  areas  of  health  care  (e.g.,  anesthesiology,  critical  care,  emergency  care, 
obstetrics,  and  surgery),  simulation  training  is  associated  with  reduced  negative  clinical 
outcomes,  which  are  linked  with  malpractice  claims  (Hanscom,  2008).  For  central  line 
placement,  a  high-risk  procedure,  Cohen  et  al.  (2010)  analyzed  costs  and  benefits  for 
simulation  training  and  found  the  benefits  were  far  greater  than  the  costs:  For  each  dol¬ 
lar  spent  on  training,  $7  was  saved  in  direct  care  costs  (Table  5.4). 

Two  related  methods  to  compare  costs  and  benefits  are  CBA  and  return  on  invest¬ 
ment  (ROI)  (e.g.,  Feldstein,  2011).  In  a  CBA,  total  benefits  are  subtracted  from  total 
costs.  If  the  CBA  is  positive,  simulation  training  results  in  cost  savings.  For  the  ROI,  the 
“return”  (total  benefits)  is  divided  by  each  dollar  “invested”  (total  costs)  on  training.  If 
the  ROI  is  greater  than  one,  simulation  training  produces  cost  savings.  A  positive  CBA 
implies  a  positive  ROI  and  vice  versa. 

Cost  values  from  the  CBA  and  ROI  performed  by  Cohen  et  al.  (2010)  are  shown  in 
Table  5.4,  comparing  the  costs  and  benefits  for  simulation  training  for  central  line 
placement  and  the  hospital  expenses  for  a  CLABSI,  a  preventable,  life-threatening, 
negative  clinical  outcome. 

The  previous  example  does  not  include  uncertainty  in  the  cost  analysis,  which  can  be 
taken  into  account  using  sensitivity  analysis  (e.g.,  Drummond,  Sculpher,  Torrance, 
O’Brien,  &  Stoddart,  2005).  A  sensitivity  analysis  could  be  incorporated  by  calculating 
the  minimum  and  maximum  cost  for  extra  hospital  days  based  on  less  expensive  care  in 
a  step-down  unit  and  more  expensive  care  in  a  critical  care  unit. 

Types  of  costs:  Fixed,  variable,  direct,  and  indirect.  Costs  can  be  categorized  as  either 
fixed  or  variable  and  direct  or  indirect  (e.g.,  Feldstein,  2011).  Fixed  costs  are  one-time 
expenses,  which  in  the  previous  example  are  the  ultrasound  machine  and  the  central  line 
simulator.  Variable  costs  are  ongoing  expenses  and  in  simulation  training  depend  on  the 
number  of  trainees  and  other  factors,  for  example,  number  of  kits  and  gowns,  simula¬ 
tion  facility  charges,  and  amount  of  salary  support  for  instructors  and  staff.  Total  costs 
are  the  sum  of  all  fixed  costs  and  all  variable  costs  (Feldstein,  2011). 

Table  5.4  excludes  indirect  costs.  From  the  perspective  of  the  health  care  facility,  direct 
costs  are  delivery  of  care  expenses  (Scott,  2009) .  Direct  costs  of  negative  clinical  outcomes 
can  be  obtained  from  published  research  and  internal  billing  records.  There  is  consider¬ 
able  uncertainty  in  estimation  of  costs  of  negative  clinical  outcomes.  For  example,  Scott 
(2009)  used  a  sensitivity  analysis  to  estimate  the  2007  direct  hospital  costs  for  CLABSI, 
which  range  from  $670  million  to  $2.68  billion  in  the  United  States.  For  the  health  care 
facility,  examples  of  indirect  costs  are  losses  because  of  decreased  rates  of  insurance  reim¬ 
bursement  due  to  frequent  adverse  clinical  outcomes,  such  as  health  care  associated 
infections,  and  the  costs  associated  with  increased  malpractice  claims.  For  the  patient, 
indirect  costs  include  lost  income,  diminished  quality  of  life,  and  mortality  (Scott,  2009). 
Indirect  costs  are  more  difficult  to  measure  and  estimate  than  direct  costs  because  they 
have  a  greater  uncertainty  (see  Andel,  Davidow,  Hollander,  &  Moreno,  2012). 
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Table  5.4.  Cost  Analysis  of  Simulation  Training  for  Central  Line  Insertions 


Costs 

Amount  (US$) 

Equipment  (e.g.,  ultrasound,  central  line  simulator,  kits,  gowns,  and 
other  items) 

$39,294 

Facility  rental 

$14,850 

Salaries:  Instructor  and  research  assistant 

$57,727 

Total  training  costs 

Benefits  (cost  savings) 

$111,916 

Simulation  training  resulted  in  137  fewer  patient  hospital  days  due  to 
a  lower  frequency  of  central  line  associated  bloodstream  infections 
(9.95  estimated  infections  with  a  predicted  average  of  13.77  additional 
hospital  days  per  infection) 

Cost-benefit  analysis  (CBA) 

$823,164 

Benefits  ($823,164)  -  Total  Costs  ($111 ,916)  The  total  benefits  minus 
the  total  costs. 

Return  on  investment 

$711,248 

CBA  /Training  Costs  =  $711 ,248  /  $111 ,916  The  return  for  each  dollar 
spent  on  training. 

7  to  1 

Note:  This  example  is  adapted  from  Cohen  et  al.  (2010),  with  simplified  values.  The  table  shows 
simulation  training  costs,  benefits,  the  cost-benefit  analysis  for  training,  and  the  return  on  invest¬ 
ment  for  training. 


Limitations 

As  with  any  other  quantitative  method,  cost  analysis  depends  upon  the  reliability 
and  validity  of  the  input  values.  The  uncertainty  of  estimating  costs  and  benefits  is 
compounded  by  the  need  to  objectively  measure  clinical  outcomes  (Lin  et  al.,  2010). 
Proxy  measures  can  be  used  to  infer  risk  for  negative  clinical  outcomes,  such  as 
suboptimal  clinical  performance  and  item  or  equipment  usage.  To  conduct  a  compre¬ 
hensive  cost  analysis  that  includes  risk,  probability  distributions  can  be  assigned  to 
values  and  modeled  (Drummond  et  al.,  2005).  Estimates  for  indirect  costs  are  sub¬ 
stantial,  possibly  in  the  hundreds  of  billions  of  dollars  in  the  United  States,  but  are  often 
overlooked  because  they  are  difficult  to  quantify  (Andel  et  al.,  2012). 

Comprehensive  reviews  of  cost  analysis  methods  can  be  found  in  Feldstein  (2011) 
and  Drummond  et  al.  (2005);  Andel  et  al.  (2012)  present  recent  data  on  different 
types  of  costs  estimates  for  medical  errors. 


CHALLENGES  AND  THE  FUTURE  OF  SIMULATION 

Preventable  adverse  clinical  outcomes  are  estimated  to  result  in  98,000  lives  lost 
every  year  and  a  projected  cost  between  $17  billion  and  $29  billion  in  the  United 
States  (Kohn,  Corrigan,  &  Donaldson,  1999);  newer  cost  analyses  are  consistent  with 
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this  range  and  indicate  total  costs  (including  adjusted  quality  of  life)  may  reach  a 
staggering  $735  billion  to  $980  billion  dollars  (Andel  et  ah,  2012).  There  is  an  obvious 
need  to  improve  patient  safety,  and  simulation  is  one  important  tool  in  this  pursuit. 
However,  numerous  challenges  prevent  health  care  from  embracing  simulation  training 
as  one  of  the  solutions. 

Among  these  challenges  is  the  strong  belief  that  the  development  of  effective  simula¬ 
tion  training  is  solely  an  engineering  problem  rather  than  a  human  factors  and  ergonom¬ 
ics  challenge.  This  sentiment  finds  its  expression  in  the  perceived  importance  of  naive 
realism  in  the  context  of  simulator  development.  Human  factors  professionals  encounter 
this  sentiment  frequently.  It  is  up  to  our  profession  to  help  developers  of  health  care 
simulators  to  move  beyond  the  fascination  with  possible  technological  solutions,  without 
understanding  the  problem,  to  the  implementation  of  solving  real  problems  using  tech¬ 
nology.  As  such,  it  will  be  one  of  our  field’s  main  tasks  to  emphasize  the  importance  of 
psychological  fidelity  rather  than  physical  fidelity  for  simulation  to  succeed. 

Another  challenge  is  that  simulator  curriculum  development  can  be  based  on  tradi¬ 
tional  clinical  curricula,  implying  very  little  need  to  perform  precise  analytical  work 
preceding  the  development  and  implementation  of  simulator-based  training  scenarios. 
For  simulation  to  succeed  in  health  care,  it  will  be  critical  to  perform  careful  and  com¬ 
prehensive  task  analyses  that  are  informed  by  the  human  factors  and  ergonomics  litera¬ 
ture.  Providing  the  simulation  training  community  with  the  results  of  task  analyses  in  a 
range  of  specialties  will  significantly  increase  the  speed  of  adoption  and  success  of  simu¬ 
lation  for  training.  In  addition,  the  results  of  task  analytical  work  will  also  improve  our 
understanding  of  human  performance,  breakdowns  of  performance,  and  the  conditions 
that  need  to  be  present  to  minimize  error  and  nonadherence  to  best  practice. 

An  additional  challenge  in  this  context  is  that  there  is  very  little  effort  devoted  to  the 
development  of  a  comprehensive  library  of  standard  scenarios  that  can  be  used  for 
health  care  workers  in  the  different  specialties.  Two  exceptions  for  standard  scenarios 
include  the  website  for  the  Society  for  Academic  Emergency  Medicine  Simulation  Case 
Library  and  publications  of  simulation  scenarios  in  the  journal  Simulation  in  Healthcare 
(see  Table  5.5).  Currently,  there  are  many  simulation  centers  nationwide  that  develop 
their  own  scenarios  to  train  health  care  workers,  which  leads  to  repetition  of  develop¬ 
ment  work  and  may  result  in  sub-optimal  simulation  trainings  because  of  resource  limi¬ 
tations.  Development  of  a  national  library  of  standard  scenarios  for  use  in  simulators  is 
highly  desirable  in  health  care  and  can  contribute  significantly  to  wider  adoption  and 
more  successful  simulator  training  in  health  care. 

As  technology  advances  over  the  next  decades,  so  will  the  sophistication  and  use 
of  simulation  training  in  health  care.  The  number  of  simulation  centers  in  hospitals 
across  the  United  States  and  the  amount  of  resources  available  for  use  of  simulation 
training  will  increase,  and  there  will  be  a  move  toward  the  use  of  simulation  as  one 
important  step  toward  an  assessment  of  knowledge,  skills,  and  abilities  of  clinicians. 
Given  the  success  of  simulation  training  in  aviation,  repeated  use  of  simulation  train¬ 
ing  and  simulation  based  certification  of  health  care  providers  is  likely  a  critical  step 
toward  improvement  of  quality  of  care  and  patient  safety. 

Finally,  with  the  advance  of  personalized  medicine,  the  use  of  simulation  to 
explore  different  procedural  approaches  and  techniques  even  before  a  procedure  is 
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Table  5.5.  Recommended  Resources 


Category 

Resource 

Reference 

Academic  books 

Institute  of  Medicine  report  on 

Kohn,  Corrigan,  &  Donaldson 

and  journals 

patient  safety 

(1999) 

“Error  in  Medicine”  (classic  article 
on  error  in  medicine) 

Leape  (1994) 

Human  Factors  (journal) 

http://hfs.sagepub.com/ 

Simulation  in  Healthcare  (journal) 

http://journals.lww.com/ 

simulationinhealthcare/pages/ 

default.aspx 

“The  Economics  of  Health  Care 

Andel,  Davidow,  Hollander,  & 

Quality  and  Medical  Errors” 
(recent  paper  on  direct  and 
indirect  cost  estimates  for 
medical  errors) 

Moreno  (2012) 

Fun  reading 

Complications:  A  Surgeon’s  Notes 
on  an  Imperfect  Science  (book) 

Gawande  (2007) 

Other  resources 

Agency  for  Healthcare  Research 
and  Quality 

http://www.  ah  rq .  gov/ 

Training  Guide:  Using  Simulation 

http://www.  ah  rq .  gov/ 

in  TeamSTEPPS  Training 

teamsteppstools/simulation/ 

index.html 

Institute  for  Healthcare 

Improvement 

http://www.ihi.org 

Simulation  for  High-Risk  Situations 

http://app.ihi.org/imap/ 
tool/#Process=f7a572 1 3-3446- 
4539-befb-66f8269c67fb 

Society  for  Academic  Emergency 
Medicine  (SAEM)  Simulation 

Case  Library 

http://www.emedu.org/simlibrary/ 

Simulation  centers 

Advanced  Simulation  Clinic, 

http://sodfiles.ahc.umn.edu/ASC 

School  of  Dentistry,  University  of 
Minnesota  (tour) 

Orientation/student. html 

Center  for  Immersive  and 
Simulation-Based  Learning, 
School  of  Medicine,  Stanford 
University 

http://cisl.stanford.edu 

Intermountain  Healthcare 

http://nursing.utah.edu/simulation- 

Simulation  Learning  Center, 
College  of  Nursing,  University  of 
Utah 

learning-center/index. php 

Johns  Hopkins  Simulation  Center 

http://www.hopkinsmedicine.org/ 

simulation_center/ 

Michael  S.  Gordon  Center  for 
Research  in  Medical  Education, 
Miller  School  of  Medicine, 
University  of  Miami 

http://www.gcrme.med.miami.edu/ 

University  of  California-Los 

Angeles  Simulation  Center 

http://www.sim.ucla.edu/ 

(continued) 
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Table  5.5.  (continued) 


Category 

Resource 

Reference 

The  Center  for  Medical  Simulation, 
Harvard  University 

http://www.harvardmedsim.org/ 

Societies 

Human  Factors  and  Ergonomics 

https://www.hfes.org/web/Default. 

Society 

asp 

International  Nursing  Association 
for  Clinical  Simulation  and 
Learning 

https://inacsl.org/ 

Society  in  Europe  for  Simulation 
Applied  to  Medicine 

http://www.sesam-web.org/ 

Society  for  Simulation  in  Health 
Care 

http://ssih.org/ 

being  performed  provides  a  great  opportunity  to  increase  the  quality  of  care  for 
patients  in  general  and  for  critically  ill  high-risk  patients  in  particular. 

At  this  point,  there  is  an  opportunity  to  accelerate  the  development  and  adoption  of 
effective  simulation  training  in  health  care  by  including  human  factors  and  ergonomics 
as  a  significant  contributor.  Human  factors  and  ergonomics  can  provide  the  theoretical 
basis  for  simulation  in  health  care  and,  as  such,  allow  leveraging  one  century  of  experi¬ 
ence  accumulated  in  aviation  simulation  training.  For  human  factors  and  ergonomics, 
this  experience  provides  a  great  opportunity  to  help  accelerate  advancement  in  patient 
safety  through  the  development  of  effective  simulation  training. 


RECOMMENDED  RESOURCES 

Table  5.5  contains  recommended  resources  for  additional  reading,  such  as  books  and 
journals,  websites  simulation  centers,  and  societies. 
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